{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 07_Train_ComplEx\n",
    "#\n",
    "# created by LuYF-Lemon-love <luyanfeng_nlp@qq.com> on February 28, 2023\n",
    "# updated by LuYF-Lemon-love <luyanfeng_nlp@qq.com> on February 28, 2023\n",
    "#\n",
    "# 该脚本展示了如何在 DRKG 上训练模型 (ComplEx), 并利用网格搜索寻找到最优参数.\n",
    "#\n",
    "# 需要的包:\n",
    "#          torch\n",
    "#          dgl, version: 0.4.3\n",
    "#          dglke\n",
    "#          numpy\n",
    "#\n",
    "# 需要的文件:\n",
    "#          ./dataset\n",
    "#\n",
    "# 源教程链接: https://github.com/gnn4dr/DRKG/blob/master/embedding_analysis/Train_embeddings.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training DRKG Using ComplEx\n",
    "\n",
    "这个 notebook 展示了如何在 DRKG 上训练模型 (ComplEx), 并利用网格搜索寻找到最优参数."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 导入需要的库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 网格搜索参数\n",
    "\n",
    "我们能使用 DGL-KE 命令训练 ComplEx 模型, 关于如何使用 DGL-KE 的更多信息请参考 https://github.com/awslabs/dgl-ke.\n",
    "\n",
    "这里我们使用两个 GPU 训练模型."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: **50**, 125, 200\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.266 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.3946738070264459\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.3950253133505583\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.6005059034973383\n",
      "[proc 0][Train](20000/100000) average loss: 0.497589855286479\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.6005255918592215\n",
      "[proc 0][Train](20000/100000) average regularization: 0.021261537669692188\n",
      "[proc 0][Train] 20000 steps take 404.442 seconds\n",
      "[proc 0]sample: 66.948, forward: 166.055, backward: 69.164, update: 99.693\n",
      "[proc 1][Train](20000/100000) average loss: 0.4977754527464509\n",
      "[proc 1][Train](20000/100000) average regularization: 0.021256930714473127\n",
      "[proc 1][Train] 20000 steps take 404.436 seconds\n",
      "[proc 1]sample: 68.146, forward: 168.197, backward: 68.494, update: 87.986\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3974239593088627\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.39724864160567525\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5638253049314023\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.563802246144414\n",
      "[proc 1][Train](40000/100000) average loss: 0.4806246320292354\n",
      "[proc 0][Train](40000/100000) average loss: 0.48052544386088847\n",
      "[proc 1][Train](40000/100000) average regularization: 0.023599395067617296\n",
      "[proc 1][Train] 20000 steps take 389.764 seconds\n",
      "[proc 1]sample: 67.441, forward: 162.312, backward: 66.716, update: 86.447\n",
      "[proc 0][Train](40000/100000) average regularization: 0.023599800239689647\n",
      "[proc 0][Train] 20000 steps take 389.777 seconds\n",
      "[proc 0]sample: 65.774, forward: 161.692, backward: 66.696, update: 93.193\n",
      "[0]Valid average MRR: 0.3286491365346257\n",
      "[0]Valid average MR: 145.2258947540448\n",
      "[0]Valid average HITS@1: 0.24236667211417987\n",
      "[0]Valid average HITS@3: 0.3590013346407365\n",
      "[0]Valid average HITS@10: 0.49485550471209894\n",
      "[proc 0]validation take 342.145 seconds:\n",
      "[1]Valid average MRR: 0.3286664921638846\n",
      "[1]Valid average MR: 142.2787575668848\n",
      "[1]Valid average HITS@1: 0.24215393205635413\n",
      "[1]Valid average HITS@3: 0.3586482088017595\n",
      "[1]Valid average HITS@10: 0.4955739256555697\n",
      "[proc 1]validation take 355.301 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3992149757146835\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.39942865969091657\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.5544972001731395\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.5544945657491684\n",
      "[proc 0][Train](60000/100000) average loss: 0.4768560879260302\n",
      "[proc 1][Train](60000/100000) average loss: 0.47696161268055437\n",
      "[proc 0][Train](60000/100000) average regularization: 0.024326558439992367\n",
      "[proc 1][Train](60000/100000) average regularization: 0.024325545762851834\n",
      "[proc 0][Train] 20000 steps take 738.178 seconds\n",
      "[proc 0]sample: 65.799, forward: 161.402, backward: 67.525, update: 84.518\n",
      "[proc 1][Train] 20000 steps take 738.184 seconds\n",
      "[proc 1]sample: 65.332, forward: 165.283, backward: 68.379, update: 82.000\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3996923238515854\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.39948238036781547\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.5486381057396531\n",
      "[proc 1][Train](80000/100000) average loss: 0.47416521490216257\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.5488062800988555\n",
      "[proc 1][Train](80000/100000) average regularization: 0.025055193265061827\n",
      "[proc 0][Train](80000/100000) average loss: 0.4741443302348256\n",
      "[proc 1][Train] 20000 steps take 388.798 seconds\n",
      "[proc 1]sample: 66.300, forward: 167.667, backward: 68.374, update: 81.294\n",
      "[proc 0][Train](80000/100000) average regularization: 0.025057262461353095\n",
      "[proc 0][Train] 20000 steps take 388.805 seconds\n",
      "[proc 0]sample: 66.759, forward: 163.994, backward: 68.679, update: 86.840\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.3991245337665081\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.543984674923122\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.39891190019994976\n",
      "[proc 1][Train](100000/100000) average loss: 0.471554604293406\n",
      "[proc 1][Train](100000/100000) average regularization: 0.025751591014675796\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.54390139567554\n",
      "[proc 1][Train] 20000 steps take 388.120 seconds\n",
      "[proc 1]sample: 67.664, forward: 165.996, backward: 70.316, update: 83.645\n",
      "[proc 0][Train](100000/100000) average loss: 0.4714066480189562\n",
      "[proc 0][Train](100000/100000) average regularization: 0.025752922503929587\n",
      "[proc 0][Train] 20000 steps take 388.127 seconds\n",
      "[proc 0]sample: 64.727, forward: 158.573, backward: 66.695, update: 84.700\n",
      "[0]Valid average MRR: 0.3469807991169364\n",
      "[0]Valid average MR: 136.86552132701422\n",
      "[0]Valid average HITS@1: 0.2557607452198072\n",
      "[0]Valid average HITS@3: 0.3815642534183145\n",
      "[0]Valid average HITS@10: 0.5230531949664978\n",
      "[proc 0]validation take 323.412 seconds:\n",
      "[1]Valid average MRR: 0.34743771338067886\n",
      "[1]Valid average MR: 134.11737608694173\n",
      "[1]Valid average HITS@1: 0.25603137746242943\n",
      "[1]Valid average HITS@3: 0.3816229393219254\n",
      "[1]Valid average HITS@10: 0.5244693817795543\n",
      "[proc 1]validation take 334.904 seconds:\n",
      "proc 1 takes 2644.205 seconds\n",
      "proc 0 takes 2644.220 seconds\n",
      "training takes 2644.7444655895233 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_0\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.3464007670047152\n",
      "Test average MR : 136.05817734258497\n",
      "Test average HITS@1 : 0.25558366301912744\n",
      "Test average HITS@3 : 0.3801776558148403\n",
      "Test average HITS@10 : 0.522271665633916\n",
      "-----------------------------------------\n",
      "testing takes 330.395 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 50.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: **50**, 125, 200\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.472 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.4036835491135716\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.4041323780894279\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5159862935677171\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5158697066307067\n",
      "[proc 0][Train](20000/100000) average loss: 0.4598349213927984\n",
      "[proc 1][Train](20000/100000) average loss: 0.460001042483747\n",
      "[proc 0][Train](20000/100000) average regularization: 0.022048135971475858\n",
      "[proc 1][Train](20000/100000) average regularization: 0.022041705748491222\n",
      "[proc 0][Train] 20000 steps take 397.838 seconds\n",
      "[proc 0]sample: 67.186, forward: 169.175, backward: 70.631, update: 90.541\n",
      "[proc 1][Train] 20000 steps take 397.817 seconds\n",
      "[proc 1]sample: 67.779, forward: 167.059, backward: 66.199, update: 87.801\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3807134515732527\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3801861324727535\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.48946106791496274\n",
      "[proc 1][Train](40000/100000) average loss: 0.43508725978285073\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.48972229764461517\n",
      "[proc 1][Train](40000/100000) average regularization: 0.027165900352224708\n",
      "[proc 0][Train](40000/100000) average loss: 0.4349542151123285\n",
      "[proc 1][Train] 20000 steps take 388.907 seconds\n",
      "[proc 1]sample: 67.543, forward: 162.103, backward: 66.947, update: 85.578\n",
      "[proc 0][Train](40000/100000) average regularization: 0.027167949507851154\n",
      "[proc 0][Train] 20000 steps take 388.917 seconds\n",
      "[proc 0]sample: 66.746, forward: 162.271, backward: 70.780, update: 87.825\n",
      "[0]Valid average MRR: 0.4849799987447362\n",
      "[0]Valid average MR: 74.33497099199215\n",
      "[0]Valid average HITS@1: 0.3822111456120281\n",
      "[0]Valid average HITS@3: 0.5370226616549545\n",
      "[0]Valid average HITS@10: 0.6792640409652994\n",
      "[proc 0]validation take 385.097 seconds:\n",
      "[1]Valid average MRR: 0.48569015919556546\n",
      "[1]Valid average MR: 72.79927752848009\n",
      "[1]Valid average HITS@1: 0.38388704658273015\n",
      "[1]Valid average HITS@3: 0.5369917675017194\n",
      "[1]Valid average HITS@10: 0.6780473521861402\n",
      "[proc 1]validation take 385.755 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.37013263680487873\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.37056097771674396\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.48172610324025156\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4817378747358918\n",
      "[proc 0][Train](60000/100000) average loss: 0.4259293700054288\n",
      "[proc 1][Train](60000/100000) average loss: 0.4261494262456894\n",
      "[proc 0][Train](60000/100000) average regularization: 0.028643805530201644\n",
      "[proc 1][Train](60000/100000) average regularization: 0.028641298781521617\n",
      "[proc 0][Train] 20000 steps take 776.729 seconds\n",
      "[proc 0]sample: 67.247, forward: 165.216, backward: 71.325, update: 86.914\n",
      "[proc 1][Train] 20000 steps take 776.739 seconds\n",
      "[proc 1]sample: 66.242, forward: 159.671, backward: 65.618, update: 84.728\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3634985173076391\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.36385999103188515\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4771553983539343\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.47690893420279024\n",
      "[proc 0][Train](80000/100000) average loss: 0.42032695777267215\n",
      "[proc 1][Train](80000/100000) average loss: 0.42038446260392665\n",
      "[proc 0][Train](80000/100000) average regularization: 0.02958626061240211\n",
      "[proc 0][Train] 20000 steps take 393.134 seconds\n",
      "[proc 0]sample: 68.598, forward: 165.360, backward: 71.972, update: 86.930\n",
      "[proc 1][Train](80000/100000) average regularization: 0.029583748880028723\n",
      "[proc 1][Train] 20000 steps take 393.136 seconds\n",
      "[proc 1]sample: 67.057, forward: 160.347, backward: 65.449, update: 85.123\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.35898695087879895\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.35865664424598215\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.47389559423923494\n",
      "[proc 1][Train](100000/100000) average loss: 0.4164412726446986\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03022756307013333\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.47398116997480394\n",
      "[proc 1][Train] 20000 steps take 391.662 seconds\n",
      "[proc 1]sample: 67.386, forward: 162.274, backward: 66.385, update: 85.355\n",
      "[proc 0][Train](100000/100000) average loss: 0.416318907135725\n",
      "[proc 0][Train](100000/100000) average regularization: 0.030229533114563674\n",
      "[proc 0][Train] 20000 steps take 391.675 seconds\n",
      "[proc 0]sample: 67.499, forward: 164.564, backward: 71.310, update: 87.786\n",
      "[0]Valid average MRR: 0.5318915831671349\n",
      "[0]Valid average MR: 77.8826741842349\n",
      "[0]Valid average HITS@1: 0.4305373971781882\n",
      "[0]Valid average HITS@3: 0.5885459225363622\n",
      "[0]Valid average HITS@10: 0.7201408182164841\n",
      "[proc 0]validation take 384.378 seconds:\n",
      "[1]Valid average MRR: 0.5318304797882802\n",
      "[1]Valid average MR: 76.20201624709752\n",
      "[1]Valid average HITS@1: 0.43049360942958115\n",
      "[1]Valid average HITS@3: 0.5882184710296411\n",
      "[1]Valid average HITS@10: 0.7199690855730404\n",
      "[proc 1]validation take 391.215 seconds:\n",
      "proc 1 takes 2739.476 seconds\n",
      "proc 0 takes 2739.499 seconds\n",
      "training takes 2740.021226644516 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_1\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5303396161604986\n",
      "Test average MR : 76.91736689432577\n",
      "Test average HITS@1 : 0.42902619555077387\n",
      "Test average HITS@3 : 0.5866914753808127\n",
      "Test average HITS@10 : 0.7186361562608524\n",
      "-----------------------------------------\n",
      "testing takes 380.779 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 50.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: **50**, 125, 200\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.285 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.3721387903645635\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.3728335249021649\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5033949164912105\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5033219119280576\n",
      "[proc 0][Train](20000/100000) average loss: 0.4377668534889817\n",
      "[proc 1][Train](20000/100000) average loss: 0.43807771853208544\n",
      "[proc 0][Train](20000/100000) average regularization: 0.02512947222351213\n",
      "[proc 1][Train](20000/100000) average regularization: 0.025137061128899223\n",
      "[proc 0][Train] 20000 steps take 394.130 seconds\n",
      "[proc 0]sample: 68.338, forward: 166.114, backward: 70.632, update: 88.300\n",
      "[proc 1][Train] 20000 steps take 394.130 seconds\n",
      "[proc 1]sample: 65.810, forward: 163.420, backward: 66.310, update: 87.909\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3482167472690344\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3486047999471426\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4741691591382027\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4742354521095753\n",
      "[proc 0][Train](40000/100000) average loss: 0.4111929531916976\n",
      "[proc 1][Train](40000/100000) average loss: 0.41142012602984906\n",
      "[proc 0][Train](40000/100000) average regularization: 0.029694129048846662\n",
      "[proc 1][Train](40000/100000) average regularization: 0.02970140441916883\n",
      "[proc 0][Train] 20000 steps take 388.444 seconds\n",
      "[proc 0]sample: 67.743, forward: 162.596, backward: 70.331, update: 87.348\n",
      "[proc 1][Train] 20000 steps take 388.445 seconds\n",
      "[proc 1]sample: 65.531, forward: 160.502, backward: 65.878, update: 85.525\n",
      "[1]Valid average MRR: 0.5527013482874825\n",
      "[1]Valid average MR: 80.99862451228066\n",
      "[1]Valid average HITS@1: 0.4509556915911397\n",
      "[1]Valid average HITS@3: 0.6123882416228031\n",
      "[1]Valid average HITS@10: 0.7398285406892419\n",
      "[proc 1]validation take 341.235 seconds:\n",
      "[0]Valid average MRR: 0.55219227318842\n",
      "[0]Valid average MR: 82.46133967968622\n",
      "[0]Valid average HITS@1: 0.4503561311761181\n",
      "[0]Valid average HITS@3: 0.6124605055292259\n",
      "[0]Valid average HITS@10: 0.7389858092280874\n",
      "[proc 0]validation take 343.555 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.34294284075051545\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.34324200584590436\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.46899544241577384\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.46903354485332965\n",
      "[proc 0][Train](60000/100000) average loss: 0.4059691415473819\n",
      "[proc 1][Train](60000/100000) average loss: 0.4061377754792571\n",
      "[proc 0][Train](60000/100000) average regularization: 0.031151280630379915\n",
      "[proc 1][Train](60000/100000) average regularization: 0.031160838504880668\n",
      "[proc 0][Train] 20000 steps take 727.563 seconds\n",
      "[proc 0]sample: 67.502, forward: 161.183, backward: 69.137, update: 85.906\n",
      "[proc 1][Train] 20000 steps take 727.563 seconds\n",
      "[proc 1]sample: 64.943, forward: 159.092, backward: 65.843, update: 84.017\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3402195170417428\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.34044317517131567\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.46651972517818213\n",
      "[proc 0][Train](80000/100000) average loss: 0.40336962121725084\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.46630398233532905\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03202075239755213\n",
      "[proc 1][Train](80000/100000) average loss: 0.40337357874661683\n",
      "[proc 0][Train] 20000 steps take 376.851 seconds\n",
      "[proc 0]sample: 65.334, forward: 156.488, backward: 69.135, update: 85.622\n",
      "[proc 1][Train](80000/100000) average regularization: 0.032029613142926246\n",
      "[proc 1][Train] 20000 steps take 376.855 seconds\n",
      "[proc 1]sample: 64.111, forward: 156.644, backward: 66.139, update: 83.255\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.33838336910903455\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.33857785623967646\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4647768531575799\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4644979115784168\n",
      "[proc 0][Train](100000/100000) average loss: 0.4015801111027598\n",
      "[proc 1][Train](100000/100000) average loss: 0.4015378839120269\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03262411688026041\n",
      "[proc 1][Train](100000/100000) average regularization: 0.032634884526766834\n",
      "[proc 0][Train] 20000 steps take 375.882 seconds\n",
      "[proc 0]sample: 64.909, forward: 156.131, backward: 69.094, update: 85.477\n",
      "[proc 1][Train] 20000 steps take 375.879 seconds\n",
      "[proc 1]sample: 63.684, forward: 156.223, backward: 65.951, update: 82.931\n",
      "[1]Valid average MRR: 0.5811299030278562\n",
      "[1]Valid average MR: 85.80649543433408\n",
      "[1]Valid average HITS@1: 0.4837494978107955\n",
      "[1]Valid average HITS@3: 0.6413347678353772\n",
      "[1]Valid average HITS@10: 0.7561845877281982\n",
      "[proc 1]validation take 340.010 seconds:\n",
      "[0]Valid average MRR: 0.5798372634952892\n",
      "[0]Valid average MR: 87.80446151331917\n",
      "[0]Valid average HITS@1: 0.48291523669444897\n",
      "[0]Valid average HITS@3: 0.6389864901672386\n",
      "[0]Valid average HITS@10: 0.7547699787546985\n",
      "[proc 0]validation take 341.144 seconds:\n",
      "proc 0 takes 2604.014 seconds\n",
      "proc 1 takes 2604.014 seconds\n",
      "training takes 2604.533656835556 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_2\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5789287067399662\n",
      "Test average MR : 88.6057933908496\n",
      "Test average HITS@1 : 0.4815160326031446\n",
      "Test average HITS@3 : 0.6387812634058982\n",
      "Test average HITS@10 : 0.7542558407157983\n",
      "-----------------------------------------\n",
      "testing takes 346.947 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 50.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 50, **125**, 200\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.298 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.4622729707673192\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.46246041693240403\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.6567395086824894\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.6567108250319957\n",
      "[proc 0][Train](20000/100000) average loss: 0.559506239721179\n",
      "[proc 1][Train](20000/100000) average loss: 0.5595856206789612\n",
      "[proc 0][Train](20000/100000) average regularization: 0.02165530597832985\n",
      "[proc 1][Train](20000/100000) average regularization: 0.021646936359955\n",
      "[proc 0][Train] 20000 steps take 399.942 seconds\n",
      "[proc 0]sample: 67.281, forward: 168.695, backward: 73.262, update: 89.958\n",
      "[proc 1][Train] 20000 steps take 399.946 seconds\n",
      "[proc 1]sample: 67.804, forward: 168.341, backward: 68.485, update: 87.642\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4323648315504193\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.43252042860090734\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.6264086033999919\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.6264804705023765\n",
      "[proc 0][Train](40000/100000) average loss: 0.5293867174074054\n",
      "[proc 0][Train](40000/100000) average regularization: 0.026902017100341617\n",
      "[proc 1][Train](40000/100000) average loss: 0.5295004496112465\n",
      "[proc 0][Train] 20000 steps take 395.673 seconds\n",
      "[proc 0]sample: 68.968, forward: 166.045, backward: 74.213, update: 86.153\n",
      "[proc 1][Train](40000/100000) average regularization: 0.02690827441411093\n",
      "[proc 1][Train] 20000 steps take 395.672 seconds\n",
      "[proc 1]sample: 66.805, forward: 163.255, backward: 67.539, update: 84.439\n",
      "[0]Valid average MRR: 0.20247008089236335\n",
      "[0]Valid average MR: 245.69680843819796\n",
      "[0]Valid average HITS@1: 0.14240140001089502\n",
      "[0]Valid average HITS@3: 0.21657610175954678\n",
      "[0]Valid average HITS@10: 0.3073010295799967\n",
      "[proc 0]validation take 371.278 seconds:\n",
      "[1]Valid average MRR: 0.2027560251079549\n",
      "[1]Valid average MR: 242.00694893672076\n",
      "[1]Valid average HITS@1: 0.1428328237673383\n",
      "[1]Valid average HITS@3: 0.2162001130351294\n",
      "[1]Valid average HITS@10: 0.3071491314680267\n",
      "[proc 1]validation take 382.187 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.43853166721463205\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.43871301800310614\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.6152534976780415\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.6150251567155123\n",
      "[proc 0][Train](60000/100000) average loss: 0.5268925823286176\n",
      "[proc 0][Train](60000/100000) average regularization: 0.026965691340062766\n",
      "[proc 1][Train](60000/100000) average loss: 0.5268690874323249\n",
      "[proc 0][Train] 20000 steps take 776.874 seconds\n",
      "[proc 0]sample: 67.317, forward: 167.273, backward: 73.075, update: 86.733\n",
      "[proc 1][Train](60000/100000) average regularization: 0.02697032896578312\n",
      "[proc 1][Train] 20000 steps take 776.874 seconds\n",
      "[proc 1]sample: 65.966, forward: 162.829, backward: 68.001, update: 84.931\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.44017644129395483\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.44040241665542124\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.6092404761075974\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.609378229561448\n",
      "[proc 1][Train](80000/100000) average loss: 0.5248214466944336\n",
      "[proc 0][Train](80000/100000) average loss: 0.5247773356050253\n",
      "[proc 1][Train](80000/100000) average regularization: 0.027077694615721702\n",
      "[proc 0][Train](80000/100000) average regularization: 0.02707496276665479\n",
      "[proc 1][Train] 20000 steps take 393.934 seconds\n",
      "[proc 1]sample: 66.507, forward: 161.759, backward: 67.247, update: 84.333\n",
      "[proc 0][Train] 20000 steps take 393.938 seconds\n",
      "[proc 0]sample: 68.111, forward: 166.276, backward: 73.515, update: 85.746\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.44068727161586285\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4404564721480012\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.6043833494573831\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.6044222230225802\n",
      "[proc 1][Train](100000/100000) average loss: 0.5225353104665875\n",
      "[proc 0][Train](100000/100000) average loss: 0.5224393475309014\n",
      "[proc 1][Train](100000/100000) average regularization: 0.02729410152165219\n",
      "[proc 1][Train] 20000 steps take 397.609 seconds\n",
      "[proc 1]sample: 66.558, forward: 164.024, backward: 67.540, update: 86.876\n",
      "[proc 0][Train](100000/100000) average regularization: 0.02729133445424959\n",
      "[proc 0][Train] 20000 steps take 397.611 seconds\n",
      "[proc 0]sample: 68.171, forward: 167.993, backward: 73.694, update: 87.456\n",
      "[0]Valid average MRR: 0.22055428159633797\n",
      "[0]Valid average MR: 232.88462847959906\n",
      "[0]Valid average HITS@1: 0.1598947268072125\n",
      "[0]Valid average HITS@3: 0.23535299885602223\n",
      "[0]Valid average HITS@10: 0.3258055510159612\n",
      "[proc 0]validation take 373.170 seconds:\n",
      "[1]Valid average MRR: 0.22079080941872903\n",
      "[1]Valid average MR: 229.41645273974\n",
      "[1]Valid average HITS@1: 0.1599583268077109\n",
      "[1]Valid average HITS@3: 0.23516754393729955\n",
      "[1]Valid average HITS@10: 0.32578630913065093\n",
      "[proc 1]validation take 390.139 seconds:\n",
      "proc 1 takes 2754.174 seconds\n",
      "proc 0 takes 2754.174 seconds\n",
      "training takes 2754.661338329315 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_3\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.21987245338871977\n",
      "Test average MR : 232.3901124903818\n",
      "Test average HITS@1 : 0.15912928903627338\n",
      "Test average HITS@3 : 0.23474195986571972\n",
      "Test average HITS@10 : 0.3247393723145645\n",
      "-----------------------------------------\n",
      "testing takes 369.186 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 125.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 50, **125**, 200\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.278 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.44241955455988646\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.44278452985435723\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5941855634897947\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5939797708749771\n",
      "[proc 0][Train](20000/100000) average loss: 0.518302559183538\n",
      "[proc 0][Train](20000/100000) average regularization: 0.02679828452626243\n",
      "[proc 1][Train](20000/100000) average loss: 0.5183821502417326\n",
      "[proc 0][Train] 20000 steps take 388.981 seconds\n",
      "[proc 0]sample: 66.636, forward: 163.106, backward: 69.217, update: 87.990\n",
      "[proc 1][Train](20000/100000) average regularization: 0.026800540223065765\n",
      "[proc 1][Train] 20000 steps take 388.963 seconds\n",
      "[proc 1]sample: 65.798, forward: 163.363, backward: 65.110, update: 85.938\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.4107633320271969\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.41054025246948006\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5413427485391498\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5415423982381821\n",
      "[proc 1][Train](40000/100000) average loss: 0.4760530402556062\n",
      "[proc 1][Train](40000/100000) average regularization: 0.032774815988633785\n",
      "[proc 0][Train](40000/100000) average loss: 0.4760413252726197\n",
      "[proc 1][Train] 20000 steps take 386.565 seconds\n",
      "[proc 1]sample: 65.053, forward: 160.229, backward: 64.830, update: 83.712\n",
      "[proc 0][Train](40000/100000) average regularization: 0.03277185236196965\n",
      "[proc 0][Train] 20000 steps take 386.575 seconds\n",
      "[proc 0]sample: 67.629, forward: 163.199, backward: 69.353, update: 86.124\n",
      "[0]Valid average MRR: 0.3655741945760113\n",
      "[0]Valid average MR: 124.00171596666122\n",
      "[0]Valid average HITS@1: 0.2727059159993463\n",
      "[0]Valid average HITS@3: 0.40179155090701096\n",
      "[0]Valid average HITS@10: 0.5444176608378275\n",
      "[proc 0]validation take 346.953 seconds:\n",
      "[1]Valid average MRR: 0.36658635793465255\n",
      "[1]Valid average MR: 121.93171588688315\n",
      "[1]Valid average HITS@1: 0.2739093131413552\n",
      "[1]Valid average HITS@3: 0.4024969868647732\n",
      "[1]Valid average HITS@10: 0.5453740713755558\n",
      "[proc 1]validation take 364.868 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3893545627698302\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.38929012204259633\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.5179410381630063\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.5183602233603597\n",
      "[proc 1][Train](60000/100000) average loss: 0.4536478004351258\n",
      "[proc 0][Train](60000/100000) average loss: 0.4538251727387309\n",
      "[proc 1][Train](60000/100000) average regularization: 0.0359126953676343\n",
      "[proc 0][Train](60000/100000) average regularization: 0.03590666862204671\n",
      "[proc 1][Train] 20000 steps take 752.456 seconds\n",
      "[proc 1]sample: 66.424, forward: 161.330, backward: 67.800, update: 85.368\n",
      "[proc 0][Train] 20000 steps take 752.453 seconds\n",
      "[proc 0]sample: 65.716, forward: 164.088, backward: 70.015, update: 83.055\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3780136486798525\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.37811697730720045\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.5061493055596947\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.5066713600590825\n",
      "[proc 1][Train](80000/100000) average loss: 0.4420814771801233\n",
      "[proc 0][Train](80000/100000) average loss: 0.44239416871368886\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03748484721872956\n",
      "[proc 1][Train](80000/100000) average regularization: 0.0374915203506127\n",
      "[proc 0][Train] 20000 steps take 388.662 seconds\n",
      "[proc 0]sample: 65.380, forward: 163.765, backward: 69.807, update: 80.708\n",
      "[proc 1][Train] 20000 steps take 388.665 seconds\n",
      "[proc 1]sample: 67.616, forward: 163.573, backward: 70.053, update: 87.163\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.36971489363461735\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3698440729036927\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4981526264026761\n",
      "[proc 1][Train](100000/100000) average loss: 0.4339337600901723\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.49858524288088085\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03848318479452282\n",
      "[proc 0][Train](100000/100000) average loss: 0.43421465783417224\n",
      "[proc 1][Train] 20000 steps take 383.154 seconds\n",
      "[proc 1]sample: 66.199, forward: 159.745, backward: 68.446, update: 85.490\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03847510024793446\n",
      "[proc 0][Train] 20000 steps take 383.160 seconds\n",
      "[proc 0]sample: 66.686, forward: 162.448, backward: 68.850, update: 83.702\n",
      "[1]Valid average MRR: 0.428429941073757\n",
      "[1]Valid average MR: 107.51746256562507\n",
      "[1]Valid average HITS@1: 0.32756014354099566\n",
      "[1]Valid average HITS@3: 0.47338226982711074\n",
      "[1]Valid average HITS@10: 0.6247744404420627\n",
      "[proc 1]validation take 332.848 seconds:\n",
      "[0]Valid average MRR: 0.4278626397349271\n",
      "[0]Valid average MR: 109.62697812823446\n",
      "[0]Valid average HITS@1: 0.3268882442664923\n",
      "[0]Valid average HITS@3: 0.4734059214468595\n",
      "[0]Valid average HITS@10: 0.6234372446478182\n",
      "[proc 0]validation take 333.109 seconds:\n",
      "proc 0 takes 2632.941 seconds\n",
      "proc 1 takes 2632.919 seconds\n",
      "training takes 2633.43878364563 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_4\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.4270659986546551\n",
      "Test average MR : 109.44471322442921\n",
      "Test average HITS@1 : 0.32639744785743957\n",
      "Test average HITS@3 : 0.4723659750641781\n",
      "Test average HITS@10 : 0.6221256051805498\n",
      "-----------------------------------------\n",
      "testing takes 328.245 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 125.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 50, **125**, 200\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.285 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.3979147426813841\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.3983674599885941\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5227897332981228\n",
      "[proc 0][Train](20000/100000) average loss: 0.46035223798006775\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5226456630498171\n",
      "[proc 0][Train](20000/100000) average regularization: 0.026101937262061982\n",
      "[proc 1][Train](20000/100000) average loss: 0.46050656154453756\n",
      "[proc 0][Train] 20000 steps take 399.225 seconds\n",
      "[proc 0]sample: 68.180, forward: 170.108, backward: 71.245, update: 89.215\n",
      "[proc 1][Train](20000/100000) average regularization: 0.026100784034468232\n",
      "[proc 1][Train] 20000 steps take 399.217 seconds\n",
      "[proc 1]sample: 65.400, forward: 168.475, backward: 71.774, update: 88.820\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.36577477189153434\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3660896362707019\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4862644734486938\n",
      "[proc 0][Train](40000/100000) average loss: 0.4260196225613356\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4861785630106926\n",
      "[proc 0][Train](40000/100000) average regularization: 0.03094189944015816\n",
      "[proc 1][Train](40000/100000) average loss: 0.4261340996414423\n",
      "[proc 0][Train] 20000 steps take 394.293 seconds\n",
      "[proc 0]sample: 68.040, forward: 167.148, backward: 70.972, update: 87.711\n",
      "[proc 1][Train](40000/100000) average regularization: 0.030940064309258015\n",
      "[proc 1][Train] 20000 steps take 394.294 seconds\n",
      "[proc 1]sample: 65.091, forward: 166.107, backward: 71.904, update: 86.075\n",
      "[0]Valid average MRR: 0.5178809428428396\n",
      "[0]Valid average MR: 91.07183227106826\n",
      "[0]Valid average HITS@1: 0.41544097619436726\n",
      "[0]Valid average HITS@3: 0.5743381271449584\n",
      "[0]Valid average HITS@10: 0.7092287683172631\n",
      "[proc 0]validation take 399.921 seconds:\n",
      "[1]Valid average MRR: 0.5171627794707777\n",
      "[1]Valid average MR: 90.14098749123296\n",
      "[1]Valid average HITS@1: 0.4148253062502979\n",
      "[1]Valid average HITS@3: 0.5729757519219376\n",
      "[1]Valid average HITS@10: 0.7090673239954514\n",
      "[proc 1]validation take 415.085 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3565928257226944\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3568464718773961\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.47802587031573057\n",
      "[proc 0][Train](60000/100000) average loss: 0.417309347961843\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4781868105441332\n",
      "[proc 0][Train](60000/100000) average regularization: 0.032139901319611815\n",
      "[proc 1][Train](60000/100000) average loss: 0.41751664114147424\n",
      "[proc 0][Train] 20000 steps take 802.879 seconds\n",
      "[proc 0]sample: 66.812, forward: 162.917, backward: 70.686, update: 87.016\n",
      "[proc 1][Train](60000/100000) average regularization: 0.032141703004855665\n",
      "[proc 1][Train] 20000 steps take 802.880 seconds\n",
      "[proc 1]sample: 64.230, forward: 163.388, backward: 69.659, update: 84.668\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.35210715006291865\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.352325111810863\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.47399260754585265\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.47414350062310695\n",
      "[proc 0][Train](80000/100000) average loss: 0.41304987878352406\n",
      "[proc 1][Train](80000/100000) average loss: 0.4132343062117696\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03278619347922504\n",
      "[proc 1][Train](80000/100000) average regularization: 0.0327881049092859\n",
      "[proc 0][Train] 20000 steps take 388.458 seconds\n",
      "[proc 0]sample: 67.116, forward: 163.040, backward: 71.424, update: 86.590\n",
      "[proc 1][Train] 20000 steps take 388.454 seconds\n",
      "[proc 1]sample: 64.085, forward: 162.124, backward: 67.565, update: 84.233\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.34927116252332924\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3490237320318818\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.47139160346984865\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4713348122417927\n",
      "[proc 1][Train](100000/100000) average loss: 0.4103313828751445\n",
      "[proc 0][Train](100000/100000) average loss: 0.4101792720615864\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03324524324759841\n",
      "[proc 0][Train](100000/100000) average regularization: 0.033241962334886195\n",
      "[proc 1][Train] 20000 steps take 392.478 seconds\n",
      "[proc 1]sample: 64.144, forward: 163.946, backward: 67.552, update: 85.071\n",
      "[proc 0][Train] 20000 steps take 392.482 seconds\n",
      "[proc 0]sample: 67.645, forward: 165.353, backward: 72.004, update: 87.178\n",
      "[1]Valid average MRR: 0.5401820964436721\n",
      "[1]Valid average MR: 93.97378402119068\n",
      "[1]Valid average HITS@1: 0.43930149737499746\n",
      "[1]Valid average HITS@3: 0.5972238299842704\n",
      "[1]Valid average HITS@10: 0.7280143268621857\n",
      "[proc 1]validation take 379.842 seconds:\n",
      "[0]Valid average MRR: 0.541082498409603\n",
      "[0]Valid average MR: 94.12529961322656\n",
      "[0]Valid average HITS@1: 0.44004671242577764\n",
      "[0]Valid average HITS@3: 0.598814484937626\n",
      "[0]Valid average HITS@10: 0.728281445770006\n",
      "[proc 0]validation take 388.545 seconds:\n",
      "proc 0 takes 2765.882 seconds\n",
      "proc 1 takes 2765.871 seconds\n",
      "training takes 2766.417323589325 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_5\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5383417420146309\n",
      "Test average MR : 95.66778226437964\n",
      "Test average HITS@1 : 0.43719230271624776\n",
      "Test average HITS@3 : 0.5955878848131175\n",
      "Test average HITS@10 : 0.726027700416051\n",
      "-----------------------------------------\n",
      "testing takes 381.308 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 125.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 50, 125, **200**\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.266 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.6795899390548468\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.6800334746748209\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.7214789621263742\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.7253130410373211\n",
      "[proc 1][Train](20000/100000) average loss: 0.7005344506353139\n",
      "[proc 1][Train](20000/100000) average regularization: 0.03664221804160625\n",
      "[proc 0][Train](20000/100000) average loss: 0.7026732580661774\n",
      "[proc 1][Train] 20000 steps take 395.517 seconds\n",
      "[proc 1]sample: 66.907, forward: 166.034, backward: 73.274, update: 87.949\n",
      "[proc 0][Train](20000/100000) average regularization: 0.03665790856145322\n",
      "[proc 0][Train] 20000 steps take 395.540 seconds\n",
      "[proc 0]sample: 67.112, forward: 165.127, backward: 70.863, update: 87.996\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.41180448238402606\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.7476052167922258\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4116197096183896\n",
      "[proc 1][Train](40000/100000) average loss: 0.5797048494845629\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.747792185986042\n",
      "[proc 1][Train](40000/100000) average regularization: 0.03081689352961257\n",
      "[proc 0][Train](40000/100000) average loss: 0.5797059478461742\n",
      "[proc 1][Train] 20000 steps take 390.215 seconds\n",
      "[proc 1]sample: 66.876, forward: 165.584, backward: 71.087, update: 81.177\n",
      "[proc 0][Train](40000/100000) average regularization: 0.030820394132006913\n",
      "[proc 0][Train] 20000 steps take 390.208 seconds\n",
      "[proc 0]sample: 67.204, forward: 161.856, backward: 71.112, update: 86.369\n",
      "[1]Valid average MRR: 0.20050455225482902\n",
      "[1]Valid average MR: 319.7038717936496\n",
      "[1]Valid average HITS@1: 0.1386620998658559\n",
      "[1]Valid average HITS@3: 0.21643844011521413\n",
      "[1]Valid average HITS@10: 0.31437044199459335\n",
      "[proc 1]validation take 387.943 seconds:\n",
      "[0]Valid average MRR: 0.19944380308366472\n",
      "[0]Valid average MR: 324.6095835376151\n",
      "[0]Valid average HITS@1: 0.13709347932668736\n",
      "[0]Valid average HITS@3: 0.21593261426158958\n",
      "[0]Valid average HITS@10: 0.3142976793593725\n",
      "[proc 0]validation take 409.702 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.4346166129395366\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.4345834392055869\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.6523034098953009\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.6526247943133116\n",
      "[proc 1][Train](60000/100000) average loss: 0.543460011343658\n",
      "[proc 0][Train](60000/100000) average loss: 0.5436041169151664\n",
      "[proc 1][Train](60000/100000) average regularization: 0.037822189294546844\n",
      "[proc 0][Train](60000/100000) average regularization: 0.037825061179324986\n",
      "[proc 1][Train] 20000 steps take 800.065 seconds\n",
      "[proc 1]sample: 66.644, forward: 163.185, backward: 71.645, update: 85.162\n",
      "[proc 0][Train] 20000 steps take 800.062 seconds\n",
      "[proc 0]sample: 65.144, forward: 161.145, backward: 70.773, update: 85.428\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.4443084955051541\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.4442908159762621\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.622499340814352\n",
      "[proc 0][Train](80000/100000) average loss: 0.5334039181515574\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.6223516459524632\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03892540283668786\n",
      "[proc 1][Train](80000/100000) average loss: 0.5333212308436632\n",
      "[proc 0][Train] 20000 steps take 394.518 seconds\n",
      "[proc 0]sample: 66.695, forward: 164.551, backward: 71.393, update: 89.311\n",
      "[proc 1][Train](80000/100000) average regularization: 0.03891694405823946\n",
      "[proc 1][Train] 20000 steps take 394.534 seconds\n",
      "[proc 1]sample: 67.791, forward: 164.029, backward: 71.717, update: 86.624\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4443179804652929\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.4442388155698776\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.6127309681296349\n",
      "[proc 0][Train](100000/100000) average loss: 0.5285244742229581\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.6125786530166865\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03857179465871304\n",
      "[proc 0][Train] 20000 steps take 411.010 seconds\n",
      "[proc 0]sample: 66.448, forward: 165.870, backward: 73.084, update: 104.890\n",
      "[proc 1][Train](100000/100000) average loss: 0.5284087342351675\n",
      "[proc 1][Train](100000/100000) average regularization: 0.038564713834412394\n",
      "[proc 1][Train] 20000 steps take 411.004 seconds\n",
      "[proc 1]sample: 68.841, forward: 167.173, backward: 72.912, update: 89.014\n",
      "[1]Valid average MRR: 0.24832751622454108\n",
      "[1]Valid average MR: 219.70621761305216\n",
      "[1]Valid average HITS@1: 0.180754066881388\n",
      "[1]Valid average HITS@3: 0.26791368474093846\n",
      "[1]Valid average HITS@10: 0.37180386362243545\n",
      "[proc 1]validation take 381.437 seconds:\n",
      "[0]Valid average MRR: 0.24727841150416585\n",
      "[0]Valid average MR: 223.68315220351909\n",
      "[0]Valid average HITS@1: 0.17945470392765703\n",
      "[0]Valid average HITS@3: 0.2674354469684589\n",
      "[0]Valid average HITS@10: 0.3707134880427085\n",
      "[proc 0]validation take 394.773 seconds:\n",
      "proc 0 takes 2786.111 seconds\n",
      "proc 1 takes 2786.100 seconds\n",
      "training takes 2786.672084569931 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_6\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.24666734719567016\n",
      "Test average MR : 221.90136493323436\n",
      "Test average HITS@1 : 0.17905683760392763\n",
      "Test average HITS@3 : 0.2667884404556814\n",
      "Test average HITS@10 : 0.3691771587326447\n",
      "-----------------------------------------\n",
      "testing takes 395.017 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 200.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 50, 125, **200**\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.185 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.4705150564789772\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.47104359132349494\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.6097168525114656\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.610343648147583\n",
      "[proc 1][Train](20000/100000) average loss: 0.540115954464674\n",
      "[proc 1][Train](20000/100000) average regularization: 0.03534979592645541\n",
      "[proc 0][Train](20000/100000) average loss: 0.5406936199128628\n",
      "[proc 1][Train] 20000 steps take 392.941 seconds\n",
      "[proc 1]sample: 66.881, forward: 164.420, backward: 71.911, update: 88.707\n",
      "[proc 0][Train](20000/100000) average regularization: 0.03536093253204599\n",
      "[proc 0][Train] 20000 steps take 392.952 seconds\n",
      "[proc 0]sample: 66.019, forward: 165.954, backward: 67.685, update: 83.783\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4568882034480572\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.4565140628442168\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5740331759467721\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5739463019728661\n",
      "[proc 0][Train](40000/100000) average loss: 0.515460689714551\n",
      "[proc 1][Train](40000/100000) average loss: 0.5152301823288202\n",
      "[proc 0][Train](40000/100000) average regularization: 0.0368594321873039\n",
      "[proc 1][Train](40000/100000) average regularization: 0.03685823528841138\n",
      "[proc 0][Train] 20000 steps take 389.057 seconds\n",
      "[proc 0]sample: 67.152, forward: 163.037, backward: 68.549, update: 83.179\n",
      "[proc 1][Train] 20000 steps take 389.067 seconds\n",
      "[proc 1]sample: 66.555, forward: 162.649, backward: 71.763, update: 85.633\n",
      "[1]Valid average MRR: 0.25328636471660754\n",
      "[1]Valid average MR: 159.38774113593496\n",
      "[1]Valid average HITS@1: 0.17933772309117033\n",
      "[1]Valid average HITS@3: 0.27283343660840137\n",
      "[1]Valid average HITS@10: 0.3881769340242549\n",
      "[proc 1]validation take 399.445 seconds:\n",
      "[0]Valid average MRR: 0.25286552513977584\n",
      "[0]Valid average MR: 163.17733017377566\n",
      "[0]Valid average HITS@1: 0.17897464182600642\n",
      "[0]Valid average HITS@3: 0.2730804325325489\n",
      "[0]Valid average HITS@10: 0.3872671188102631\n",
      "[proc 0]validation take 403.057 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.44940509981662036\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.4489799477264285\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.5626755549713969\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.5626494586244225\n",
      "[proc 0][Train](60000/100000) average loss: 0.5060403276130557\n",
      "[proc 1][Train](60000/100000) average loss: 0.5058147031664848\n",
      "[proc 0][Train](60000/100000) average regularization: 0.037435303161479534\n",
      "[proc 1][Train](60000/100000) average regularization: 0.03743457759208977\n",
      "[proc 0][Train] 20000 steps take 786.008 seconds\n",
      "[proc 0]sample: 65.738, forward: 159.871, backward: 68.807, update: 83.356\n",
      "[proc 1][Train] 20000 steps take 786.009 seconds\n",
      "[proc 1]sample: 65.008, forward: 160.293, backward: 71.659, update: 84.125\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.44107141865938904\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.44149809850603344\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.5542839025944472\n",
      "[proc 1][Train](80000/100000) average loss: 0.4976776608556509\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.553935740865767\n",
      "[proc 1][Train](80000/100000) average regularization: 0.0379016105171293\n",
      "[proc 0][Train](80000/100000) average loss: 0.49771691990047695\n",
      "[proc 1][Train] 20000 steps take 394.500 seconds\n",
      "[proc 1]sample: 63.905, forward: 155.308, backward: 70.292, update: 86.928\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03790293545424938\n",
      "[proc 0][Train] 20000 steps take 394.514 seconds\n",
      "[proc 0]sample: 66.632, forward: 156.093, backward: 70.839, update: 99.694\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.43095310878902676\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4313541255414486\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.5449598327115178\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.5449102472022176\n",
      "[proc 1][Train](100000/100000) average loss: 0.4879564708173275\n",
      "[proc 1][Train](100000/100000) average regularization: 0.039087886805459854\n",
      "[proc 0][Train](100000/100000) average loss: 0.48813218648135664\n",
      "[proc 1][Train] 20000 steps take 388.585 seconds\n",
      "[proc 1]sample: 64.145, forward: 157.912, backward: 66.588, update: 82.699\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03909242820441723\n",
      "[proc 0][Train] 20000 steps take 388.577 seconds\n",
      "[proc 0]sample: 67.463, forward: 161.925, backward: 71.855, update: 87.039\n",
      "[1]Valid average MRR: 0.3213580182020486\n",
      "[1]Valid average MR: 135.4179610096897\n",
      "[1]Valid average HITS@1: 0.23454448885650667\n",
      "[1]Valid average HITS@3: 0.35075277310581043\n",
      "[1]Valid average HITS@10: 0.485356503265081\n",
      "[proc 1]validation take 380.187 seconds:\n",
      "[0]Valid average MRR: 0.3207755248926456\n",
      "[0]Valid average MR: 138.59269965135914\n",
      "[0]Valid average HITS@1: 0.23358936645421366\n",
      "[0]Valid average HITS@3: 0.35099689491747016\n",
      "[0]Valid average HITS@10: 0.4851657405894209\n",
      "[proc 0]validation take 405.330 seconds:\n",
      "proc 0 takes 2756.437 seconds\n",
      "proc 1 takes 2756.434 seconds\n",
      "training takes 2756.9580025672913 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_7\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.32109979420406537\n",
      "Test average MR : 138.23956638090115\n",
      "Test average HITS@1 : 0.23418189122752064\n",
      "Test average HITS@3 : 0.3514132795849023\n",
      "Test average HITS@10 : 0.4849700729280865\n",
      "-----------------------------------------\n",
      "testing takes 378.639 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 200.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: **200**, 400\n",
    "\n",
    "- gamma: 50, 125, **200**\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.218 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.4175164299070835\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.41777187396883964\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5875717149153352\n",
      "[proc 0][Train](20000/100000) average loss: 0.5025440722703933\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5871140824630856\n",
      "[proc 0][Train](20000/100000) average regularization: 0.03429129828009755\n",
      "[proc 1][Train](20000/100000) average loss: 0.5024429780200124\n",
      "[proc 0][Train] 20000 steps take 390.820 seconds\n",
      "[proc 0]sample: 66.034, forward: 161.835, backward: 69.893, update: 86.800\n",
      "[proc 1][Train](20000/100000) average regularization: 0.03428591730957851\n",
      "[proc 1][Train] 20000 steps take 390.814 seconds\n",
      "[proc 1]sample: 66.884, forward: 161.167, backward: 72.534, update: 86.473\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.37718577920496466\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.37718993356674907\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5182541379988194\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5197603268444538\n",
      "[proc 0][Train](40000/100000) average loss: 0.4477199586331844\n",
      "[proc 1][Train](40000/100000) average loss: 0.44847513013333085\n",
      "[proc 0][Train](40000/100000) average regularization: 0.03787912717927247\n",
      "[proc 1][Train](40000/100000) average regularization: 0.037872798366099594\n",
      "[proc 0][Train] 20000 steps take 390.248 seconds\n",
      "[proc 0]sample: 64.355, forward: 155.059, backward: 70.046, update: 95.396\n",
      "[proc 1][Train] 20000 steps take 390.240 seconds\n",
      "[proc 1]sample: 67.465, forward: 161.788, backward: 74.018, update: 86.042\n",
      "[1]Valid average MRR: 0.43697661564806006\n",
      "[1]Valid average MR: 103.26278965251912\n",
      "[1]Valid average HITS@1: 0.3297595620229203\n",
      "[1]Valid average HITS@3: 0.48968724677747744\n",
      "[1]Valid average HITS@10: 0.6418216360132646\n",
      "[proc 1]validation take 401.040 seconds:\n",
      "[0]Valid average MRR: 0.4383631596632666\n",
      "[0]Valid average MR: 104.44398254072016\n",
      "[0]Valid average HITS@1: 0.331607152584845\n",
      "[0]Valid average HITS@3: 0.49034087813912947\n",
      "[0]Valid average HITS@10: 0.642929127853135\n",
      "[proc 0]validation take 423.738 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3658826205730438\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3654608713105321\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4980348394662142\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.49792711048424243\n",
      "[proc 0][Train](60000/100000) average loss: 0.43169399092793465\n",
      "[proc 1][Train](60000/100000) average loss: 0.4319587301030755\n",
      "[proc 0][Train](60000/100000) average regularization: 0.037077105994522574\n",
      "[proc 1][Train](60000/100000) average regularization: 0.03707015690933913\n",
      "[proc 0][Train] 20000 steps take 804.748 seconds\n",
      "[proc 0]sample: 63.514, forward: 157.407, backward: 67.684, update: 78.416\n",
      "[proc 1][Train] 20000 steps take 804.745 seconds\n",
      "[proc 1]sample: 65.577, forward: 159.679, backward: 72.815, update: 82.670\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.357907187590003\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3581986702054739\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4859751426100731\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4860459415525198\n",
      "[proc 0][Train](80000/100000) average loss: 0.4219411651417613\n",
      "[proc 1][Train](80000/100000) average loss: 0.4221223058000207\n",
      "[proc 1][Train](80000/100000) average regularization: 0.036200555891543626\n",
      "[proc 0][Train](80000/100000) average regularization: 0.036204852791316806\n",
      "[proc 1][Train] 20000 steps take 373.306 seconds\n",
      "[proc 1]sample: 65.118, forward: 156.697, backward: 71.415, update: 79.812\n",
      "[proc 0][Train] 20000 steps take 373.307 seconds\n",
      "[proc 0]sample: 63.483, forward: 155.414, backward: 67.418, update: 76.760\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.35182101767212154\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.3521366975992918\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.47881763785779474\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4788080417379737\n",
      "[proc 1][Train](100000/100000) average loss: 0.4154771676659584\n",
      "[proc 0][Train](100000/100000) average loss: 0.4153145296871662\n",
      "[proc 0][Train](100000/100000) average regularization: 0.036147897508367895\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03614223665930331\n",
      "[proc 0][Train] 20000 steps take 390.600 seconds\n",
      "[proc 0]sample: 64.314, forward: 158.234, backward: 67.592, update: 77.949\n",
      "[proc 1][Train] 20000 steps take 390.600 seconds\n",
      "[proc 1]sample: 67.070, forward: 162.455, backward: 74.571, update: 86.227\n",
      "[1]Valid average MRR: 0.49600111565653976\n",
      "[1]Valid average MR: 96.99782441422609\n",
      "[1]Valid average HITS@1: 0.3914760617471418\n",
      "[1]Valid average HITS@3: 0.5520199922373465\n",
      "[1]Valid average HITS@10: 0.6918124433973185\n",
      "[proc 1]validation take 387.601 seconds:\n",
      "[0]Valid average MRR: 0.49612402287679197\n",
      "[0]Valid average MR: 97.00388475785803\n",
      "[0]Valid average HITS@1: 0.3909986653592635\n",
      "[0]Valid average HITS@3: 0.5527795936155145\n",
      "[0]Valid average HITS@10: 0.6922018848395708\n",
      "[proc 0]validation take 421.272 seconds:\n",
      "proc 0 takes 2770.995 seconds\n",
      "proc 1 takes 2770.979 seconds\n",
      "training takes 2771.505744457245 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_8\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.4943966262519654\n",
      "Test average MR : 97.09381915741163\n",
      "Test average HITS@1 : 0.38976521377939083\n",
      "Test average HITS@3 : 0.5509730554212602\n",
      "Test average HITS@10 : 0.6897968772343164\n",
      "-----------------------------------------\n",
      "testing takes 401.776 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 200 \\\n",
    "--gamma 200.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 10\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: **50**, 125, 200\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.492 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.4208501549899578\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.4211138929069042\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.6164896643519402\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.6164194512426853\n",
      "[proc 1][Train](20000/100000) average loss: 0.5186699097439647\n",
      "[proc 1][Train](20000/100000) average regularization: 0.0192578497468523\n",
      "[proc 0][Train](20000/100000) average loss: 0.5187666721552611\n",
      "[proc 1][Train] 20000 steps take 516.446 seconds\n",
      "[proc 1]sample: 69.530, forward: 241.335, backward: 72.208, update: 121.284\n",
      "[proc 0][Train](20000/100000) average regularization: 0.019251299895517878\n",
      "[proc 0][Train] 20000 steps take 516.461 seconds\n",
      "[proc 0]sample: 70.691, forward: 235.134, backward: 72.224, update: 135.066\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.42302924412786963\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.4232219414189458\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5824820770293474\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5822729946017265\n",
      "[proc 1][Train](40000/100000) average loss: 0.5027556606605649\n",
      "[proc 0][Train](40000/100000) average loss: 0.5027474682226777\n",
      "[proc 1][Train](40000/100000) average regularization: 0.022106441703625024\n",
      "[proc 1][Train] 20000 steps take 510.183 seconds\n",
      "[proc 1]sample: 70.316, forward: 237.086, backward: 73.046, update: 125.227\n",
      "[proc 0][Train](40000/100000) average regularization: 0.02209216350093484\n",
      "[proc 0][Train] 20000 steps take 510.181 seconds\n",
      "[proc 0]sample: 69.826, forward: 228.954, backward: 70.521, update: 130.803\n",
      "[0]Valid average MRR: 0.32000109649092123\n",
      "[0]Valid average MR: 153.84594432641498\n",
      "[0]Valid average HITS@1: 0.23731410361170127\n",
      "[0]Valid average HITS@3: 0.3487191534564471\n",
      "[0]Valid average HITS@10: 0.47814866263550687\n",
      "[proc 0]validation take 346.176 seconds:\n",
      "[1]Valid average MRR: 0.32026595883560344\n",
      "[1]Valid average MR: 150.1371640439339\n",
      "[1]Valid average HITS@1: 0.23732610634835247\n",
      "[1]Valid average HITS@3: 0.3492036470852598\n",
      "[1]Valid average HITS@10: 0.47946641971441606\n",
      "[proc 1]validation take 347.939 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.42199694098234175\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.4218766845092177\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.5704281564593315\n",
      "[proc 0][Train](60000/100000) average loss: 0.49621254861950875\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.5707745742246508\n",
      "[proc 0][Train](60000/100000) average regularization: 0.022996410382725298\n",
      "[proc 1][Train](60000/100000) average loss: 0.49632562946975234\n",
      "[proc 0][Train] 20000 steps take 869.500 seconds\n",
      "[proc 0]sample: 69.811, forward: 240.954, backward: 73.133, update: 134.334\n",
      "[proc 1][Train](60000/100000) average regularization: 0.02300909450557083\n",
      "[proc 1][Train] 20000 steps take 869.511 seconds\n",
      "[proc 1]sample: 69.864, forward: 238.765, backward: 72.776, update: 131.712\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.4211755986973643\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.4211231008008122\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.5637637173712253\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.5637539300009609\n",
      "[proc 1][Train](80000/100000) average loss: 0.4924434091106057\n",
      "[proc 0][Train](80000/100000) average loss: 0.49246476432085035\n",
      "[proc 1][Train](80000/100000) average regularization: 0.0238355431959033\n",
      "[proc 0][Train](80000/100000) average regularization: 0.023822766692284496\n",
      "[proc 1][Train] 20000 steps take 538.674 seconds\n",
      "[proc 1]sample: 72.899, forward: 245.620, backward: 76.490, update: 139.042\n",
      "[proc 0][Train] 20000 steps take 538.681 seconds\n",
      "[proc 0]sample: 69.218, forward: 252.778, backward: 74.513, update: 133.676\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4195519644588232\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.5586452629178762\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.4194745745822787\n",
      "[proc 0][Train](100000/100000) average loss: 0.48909861356317996\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.5586029409870505\n",
      "[proc 0][Train](100000/100000) average regularization: 0.024664324818737805\n",
      "[proc 0][Train] 20000 steps take 552.524 seconds\n",
      "[proc 0]sample: 71.132, forward: 249.255, backward: 75.144, update: 136.482\n",
      "[proc 1][Train](100000/100000) average loss: 0.4890387577757239\n",
      "[proc 1][Train](100000/100000) average regularization: 0.024675833983626216\n",
      "[proc 1][Train] 20000 steps take 552.531 seconds\n",
      "[proc 1]sample: 70.866, forward: 239.143, backward: 75.000, update: 152.333\n",
      "[1]Valid average MRR: 0.3457875705998156\n",
      "[1]Valid average MR: 138.9755714742913\n",
      "[1]Valid average HITS@1: 0.25656931572890634\n",
      "[1]Valid average HITS@3: 0.3794575675657272\n",
      "[1]Valid average HITS@10: 0.5174693749702091\n",
      "[proc 1]validation take 343.880 seconds:\n",
      "[0]Valid average MRR: 0.34551373128746576\n",
      "[0]Valid average MR: 142.5985046576238\n",
      "[0]Valid average HITS@1: 0.25680598681701805\n",
      "[0]Valid average HITS@3: 0.38010023424306805\n",
      "[0]Valid average HITS@10: 0.5153007027292041\n",
      "[proc 0]validation take 345.289 seconds:\n",
      "proc 0 takes 3332.637 seconds\n",
      "proc 1 takes 3332.626 seconds\n",
      "training takes 3333.1663794517517 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_9\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.3442697591843235\n",
      "Test average MR : 141.9725259946751\n",
      "Test average HITS@1 : 0.25572836160346457\n",
      "Test average HITS@3 : 0.37791014388146293\n",
      "Test average HITS@10 : 0.5141464145393138\n",
      "-----------------------------------------\n",
      "testing takes 376.025 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 50.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 11\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: **50**, 125, 200\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.330 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.36694249092042447\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.3667017302110791\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4922269703939557\n",
      "[proc 0][Train](20000/100000) average loss: 0.42958473064005376\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.4935086443811655\n",
      "[proc 0][Train](20000/100000) average regularization: 0.025410874880508347\n",
      "[proc 1][Train](20000/100000) average loss: 0.43010518739670517\n",
      "[proc 0][Train] 20000 steps take 644.566 seconds\n",
      "[proc 0]sample: 71.006, forward: 237.863, backward: 78.493, update: 256.854\n",
      "[proc 1][Train](20000/100000) average regularization: 0.025362817253194225\n",
      "[proc 1][Train] 20000 steps take 644.568 seconds\n",
      "[proc 1]sample: 70.751, forward: 240.214, backward: 76.343, update: 188.770\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3382858243107796\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3382926537245512\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4598234598085284\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4604965278729796\n",
      "[proc 0][Train](40000/100000) average loss: 0.399054642149806\n",
      "[proc 0][Train](40000/100000) average regularization: 0.03144081293307245\n",
      "[proc 1][Train](40000/100000) average loss: 0.39939459074884653\n",
      "[proc 0][Train] 20000 steps take 658.352 seconds\n",
      "[proc 0]sample: 70.461, forward: 228.904, backward: 75.987, update: 282.659\n",
      "[proc 1][Train](40000/100000) average regularization: 0.031438254146464166\n",
      "[proc 1][Train] 20000 steps take 658.353 seconds\n",
      "[proc 1]sample: 69.528, forward: 234.969, backward: 73.269, update: 123.294\n",
      "[0]Valid average MRR: 0.5723128224360386\n",
      "[0]Valid average MR: 72.27440826387755\n",
      "[0]Valid average HITS@1: 0.4753942637685896\n",
      "[0]Valid average HITS@3: 0.6296133627499046\n",
      "[0]Valid average HITS@10: 0.7492305387590565\n",
      "[proc 0]validation take 414.055 seconds:\n",
      "[1]Valid average MRR: 0.5740013031879478\n",
      "[1]Valid average MR: 71.51201849418142\n",
      "[1]Valid average HITS@1: 0.47731807132108106\n",
      "[1]Valid average HITS@3: 0.6315020734455967\n",
      "[1]Valid average HITS@10: 0.7502332200712257\n",
      "[proc 1]validation take 422.771 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.32794131312668323\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3273975600719452\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4511931746914983\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.45147815131098035\n",
      "[proc 0][Train](60000/100000) average loss: 0.3892953673392534\n",
      "[proc 1][Train](60000/100000) average loss: 0.3897097322136164\n",
      "[proc 0][Train](60000/100000) average regularization: 0.03308548070862889\n",
      "[proc 1][Train](60000/100000) average regularization: 0.03309656512513757\n",
      "[proc 0][Train] 20000 steps take 970.388 seconds\n",
      "[proc 0]sample: 67.158, forward: 239.888, backward: 71.940, update: 166.874\n",
      "[proc 1][Train] 20000 steps take 970.384 seconds\n",
      "[proc 1]sample: 67.603, forward: 244.229, backward: 72.339, update: 117.654\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3216807069391012\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3221198323830962\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4466185670569539\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4467284337118268\n",
      "[proc 0][Train](80000/100000) average loss: 0.3841496369123459\n",
      "[proc 1][Train](80000/100000) average loss: 0.38442413301318884\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03403302989397198\n",
      "[proc 1][Train](80000/100000) average regularization: 0.03404584073927253\n",
      "[proc 0][Train] 20000 steps take 494.100 seconds\n",
      "[proc 0]sample: 66.019, forward: 240.904, backward: 70.783, update: 113.202\n",
      "[proc 1][Train] 20000 steps take 494.098 seconds\n",
      "[proc 1]sample: 66.601, forward: 243.862, backward: 71.808, update: 111.223\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.31825603758394716\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3178728400334716\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4437322879225016\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.44359355682581664\n",
      "[proc 1][Train](100000/100000) average loss: 0.38099416267871855\n",
      "[proc 0][Train](100000/100000) average loss: 0.38073319829404356\n",
      "[proc 1][Train](100000/100000) average regularization: 0.034725197971612214\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03471145992521196\n",
      "[proc 1][Train] 20000 steps take 491.893 seconds\n",
      "[proc 1]sample: 66.372, forward: 245.933, backward: 72.085, update: 106.436\n",
      "[proc 0][Train] 20000 steps take 491.895 seconds\n",
      "[proc 0]sample: 65.809, forward: 242.742, backward: 71.456, update: 108.092\n",
      "[0]Valid average MRR: 0.5968819649446984\n",
      "[0]Valid average MR: 85.82162458462712\n",
      "[0]Valid average HITS@1: 0.5054441085144632\n",
      "[0]Valid average HITS@3: 0.652652938933377\n",
      "[0]Valid average HITS@10: 0.7615793702674729\n",
      "[proc 0]validation take 395.192 seconds:\n",
      "[1]Valid average MRR: 0.5982212907615312\n",
      "[1]Valid average MR: 83.6159597431515\n",
      "[1]Valid average HITS@1: 0.5072247151991393\n",
      "[1]Valid average HITS@3: 0.653577970406586\n",
      "[1]Valid average HITS@10: 0.7624491852618533\n",
      "[proc 1]validation take 404.229 seconds:\n",
      "proc 1 takes 3663.526 seconds\n",
      "proc 0 takes 3663.530 seconds\n",
      "training takes 3664.041381597519 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_10\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.596113207860975\n",
      "Test average MR : 86.42505294265851\n",
      "Test average HITS@1 : 0.5044005393001355\n",
      "Test average HITS@3 : 0.652245041094398\n",
      "Test average HITS@10 : 0.7612183961268445\n",
      "-----------------------------------------\n",
      "testing takes 385.153 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 50.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: **50**, 125, 200\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.393 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.34040715615302325\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4896481912255287\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.3398924427345395\n",
      "[proc 0][Train](20000/100000) average loss: 0.41502767363041637\n",
      "[proc 0][Train](20000/100000) average regularization: 0.030087162997870474\n",
      "[proc 0][Train] 20000 steps take 663.621 seconds\n",
      "[proc 0]sample: 71.498, forward: 237.532, backward: 78.913, update: 275.338\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.49269460367560386\n",
      "[proc 1][Train](20000/100000) average loss: 0.41629352322518826\n",
      "[proc 1][Train](20000/100000) average regularization: 0.029990192736104655\n",
      "[proc 1][Train] 20000 steps take 663.634 seconds\n",
      "[proc 1]sample: 68.766, forward: 240.774, backward: 72.755, update: 111.551\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.30998750598132613\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3094432883888483\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.44574856453984973\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.44611446523964404\n",
      "[proc 0][Train](40000/100000) average loss: 0.377595926360786\n",
      "[proc 1][Train](40000/100000) average loss: 0.3780509855851531\n",
      "[proc 0][Train](40000/100000) average regularization: 0.034209971454180775\n",
      "[proc 1][Train](40000/100000) average regularization: 0.034207499074563386\n",
      "[proc 0][Train] 20000 steps take 548.904 seconds\n",
      "[proc 0]sample: 72.425, forward: 249.509, backward: 75.063, update: 151.492\n",
      "[proc 1][Train] 20000 steps take 548.884 seconds\n",
      "[proc 1]sample: 68.755, forward: 245.368, backward: 72.574, update: 123.440\n",
      "[0]Valid average MRR: 0.6029512011293514\n",
      "[0]Valid average MR: 101.02947445116304\n",
      "[0]Valid average HITS@1: 0.5133940731056272\n",
      "[0]Valid average HITS@3: 0.6584511358065044\n",
      "[0]Valid average HITS@10: 0.7634281200631912\n",
      "[proc 0]validation take 400.744 seconds:\n",
      "[1]Valid average MRR: 0.603312328829706\n",
      "[1]Valid average MR: 100.17115289022655\n",
      "[1]Valid average HITS@1: 0.513550596839102\n",
      "[1]Valid average HITS@3: 0.6590288511953806\n",
      "[1]Valid average HITS@10: 0.7641140701498736\n",
      "[proc 1]validation take 424.100 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.30396713805645703\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.304239078655839\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4401352288633585\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.43999638211280107\n",
      "[proc 0][Train](60000/100000) average loss: 0.37205118340700866\n",
      "[proc 1][Train](60000/100000) average loss: 0.3721177303791046\n",
      "[proc 0][Train](60000/100000) average regularization: 0.03561559458598495\n",
      "[proc 1][Train](60000/100000) average regularization: 0.03561580812558532\n",
      "[proc 0][Train] 20000 steps take 954.575 seconds\n",
      "[proc 0]sample: 71.706, forward: 251.925, backward: 74.660, update: 131.207\n",
      "[proc 1][Train] 20000 steps take 954.576 seconds\n",
      "[proc 1]sample: 68.996, forward: 249.396, backward: 73.048, update: 118.798\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.30142840817719696\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.30160654330402614\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.4372538557946682\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4369971250846982\n",
      "[proc 1][Train](80000/100000) average loss: 0.369430199688673\n",
      "[proc 0][Train](80000/100000) average loss: 0.3692127666220069\n",
      "[proc 1][Train](80000/100000) average regularization: 0.036413922607526183\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03641236924100667\n",
      "[proc 1][Train] 20000 steps take 516.805 seconds\n",
      "[proc 1]sample: 68.920, forward: 249.983, backward: 73.084, update: 111.124\n",
      "[proc 0][Train] 20000 steps take 516.807 seconds\n",
      "[proc 0]sample: 70.121, forward: 240.403, backward: 72.507, update: 129.011\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.299819611261785\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.2998838715344667\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4349161361843348\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4353468943580985\n",
      "[proc 0][Train](100000/100000) average loss: 0.367367873673141\n",
      "[proc 1][Train](100000/100000) average loss: 0.3676153830394149\n",
      "[proc 0][Train](100000/100000) average regularization: 0.036944782922044395\n",
      "[proc 0][Train] 20000 steps take 511.706 seconds\n",
      "[proc 0]sample: 70.939, forward: 237.726, backward: 72.023, update: 126.434\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03694666230417788\n",
      "[proc 1][Train] 20000 steps take 511.709 seconds\n",
      "[proc 1]sample: 67.955, forward: 247.665, backward: 71.465, update: 120.672\n",
      "[0]Valid average MRR: 0.6225582943375586\n",
      "[0]Valid average MR: 113.25173299014\n",
      "[0]Valid average HITS@1: 0.5392050716347987\n",
      "[0]Valid average HITS@3: 0.6751477637958272\n",
      "[0]Valid average HITS@10: 0.7693318625047666\n",
      "[proc 0]validation take 427.253 seconds:\n",
      "[1]Valid average MRR: 0.6232425193927783\n",
      "[1]Valid average MR: 112.21650312889409\n",
      "[1]Valid average HITS@1: 0.5398244550821547\n",
      "[1]Valid average HITS@3: 0.675854743049361\n",
      "[1]Valid average HITS@10: 0.7700960798599998\n",
      "[proc 1]validation take 427.843 seconds:\n",
      "proc 1 takes 3623.452 seconds\n",
      "proc 0 takes 3623.459 seconds\n",
      "training takes 3623.964634656906 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_11\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.6205933890482063\n",
      "Test average MR : 112.74029838550426\n",
      "Test average HITS@1 : 0.5368538782625275\n",
      "Test average HITS@3 : 0.673296131611023\n",
      "Test average HITS@10 : 0.7684822650605692\n",
      "-----------------------------------------\n",
      "testing takes 412.977 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 50.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 50, **125**, 200\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.359 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.39655860163867473\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.3966435058251023\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.6066095931857824\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.6065218027681112\n",
      "[proc 1][Train](20000/100000) average loss: 0.5015840974092484\n",
      "[proc 0][Train](20000/100000) average loss: 0.5015826541423798\n",
      "[proc 1][Train](20000/100000) average regularization: 0.022417273386765735\n",
      "[proc 0][Train](20000/100000) average regularization: 0.022423231460060925\n",
      "[proc 1][Train] 20000 steps take 518.153 seconds\n",
      "[proc 1]sample: 70.624, forward: 249.923, backward: 73.247, update: 121.629\n",
      "[proc 0][Train] 20000 steps take 518.158 seconds\n",
      "[proc 0]sample: 70.310, forward: 246.082, backward: 72.241, update: 126.530\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.38618591099977495\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3860544109135866\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5514856947302819\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5519533098548651\n",
      "[proc 0][Train](40000/100000) average loss: 0.4688358028933406\n",
      "[proc 1][Train](40000/100000) average loss: 0.4690038604274392\n",
      "[proc 0][Train](40000/100000) average regularization: 0.027287512496486307\n",
      "[proc 1][Train](40000/100000) average regularization: 0.027276689297612757\n",
      "[proc 0][Train] 20000 steps take 498.704 seconds\n",
      "[proc 0]sample: 67.930, forward: 240.023, backward: 69.142, update: 116.795\n",
      "[proc 1][Train] 20000 steps take 498.705 seconds\n",
      "[proc 1]sample: 71.615, forward: 232.479, backward: 69.043, update: 123.936\n",
      "[0]Valid average MRR: 0.35020636120875587\n",
      "[0]Valid average MR: 156.0497902707414\n",
      "[0]Valid average HITS@1: 0.2625769461240943\n",
      "[0]Valid average HITS@3: 0.38400882497140054\n",
      "[0]Valid average HITS@10: 0.5176329193223294\n",
      "[proc 0]validation take 405.864 seconds:\n",
      "[1]Valid average MRR: 0.3496894494961308\n",
      "[1]Valid average MR: 153.04592562833233\n",
      "[1]Valid average HITS@1: 0.26187720026965006\n",
      "[1]Valid average HITS@3: 0.3833695363516891\n",
      "[1]Valid average HITS@10: 0.5174829936604997\n",
      "[proc 1]validation take 416.401 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.380860236351192\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.38098383489698173\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.5340611803680658\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.5338112956687808\n",
      "[proc 1][Train](60000/100000) average loss: 0.45746070824712515\n",
      "[proc 0][Train](60000/100000) average loss: 0.45739756538569926\n",
      "[proc 1][Train](60000/100000) average regularization: 0.029032618701457978\n",
      "[proc 0][Train](60000/100000) average regularization: 0.02904610770912841\n",
      "[proc 1][Train] 20000 steps take 920.690 seconds\n",
      "[proc 1]sample: 70.083, forward: 241.445, backward: 71.694, update: 118.945\n",
      "[proc 0][Train] 20000 steps take 920.692 seconds\n",
      "[proc 0]sample: 68.569, forward: 237.974, backward: 70.467, update: 120.364\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3771102183073759\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.37717472293823956\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.5216279278606176\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.5214681806355714\n",
      "[proc 1][Train](80000/100000) average loss: 0.4493690729856491\n",
      "[proc 0][Train](80000/100000) average loss: 0.44932145172804594\n",
      "[proc 1][Train](80000/100000) average regularization: 0.030481665062904357\n",
      "[proc 1][Train] 20000 steps take 543.726 seconds\n",
      "[proc 1]sample: 70.264, forward: 242.779, backward: 74.304, update: 154.973\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03049458790300414\n",
      "[proc 0][Train] 20000 steps take 543.731 seconds\n",
      "[proc 0]sample: 69.615, forward: 239.456, backward: 72.413, update: 120.563\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.37399355743676427\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.3740779965892434\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.5115622472941875\n",
      "[proc 1][Train](100000/100000) average loss: 0.44277790238559245\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.5114539047762752\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03171812488026917\n",
      "[proc 0][Train](100000/100000) average loss: 0.44276595072895286\n",
      "[proc 1][Train] 20000 steps take 577.283 seconds\n",
      "[proc 1]sample: 70.442, forward: 236.205, backward: 75.902, update: 194.437\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03172896076003089\n",
      "[proc 0][Train] 20000 steps take 577.281 seconds\n",
      "[proc 0]sample: 70.857, forward: 235.667, backward: 73.251, update: 121.758\n",
      "[0]Valid average MRR: 0.3823081701644151\n",
      "[0]Valid average MR: 133.4291074249605\n",
      "[0]Valid average HITS@1: 0.2886773437925587\n",
      "[0]Valid average HITS@3: 0.42136514681048104\n",
      "[0]Valid average HITS@10: 0.5626838535708449\n",
      "[proc 0]validation take 383.852 seconds:\n",
      "[1]Valid average MRR: 0.38265855488797723\n",
      "[1]Valid average MR: 131.73779595116338\n",
      "[1]Valid average HITS@1: 0.28911117617818693\n",
      "[1]Valid average HITS@3: 0.4207528411992619\n",
      "[1]Valid average HITS@10: 0.5635073574974294\n",
      "[proc 1]validation take 417.298 seconds:\n",
      "proc 1 takes 3475.855 seconds\n",
      "proc 0 takes 3475.860 seconds\n",
      "training takes 3476.4166390895844 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_12\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.3817099857366403\n",
      "Test average MR : 134.11879243073193\n",
      "Test average HITS@1 : 0.2881493561764165\n",
      "Test average HITS@3 : 0.4206166542963563\n",
      "Test average HITS@10 : 0.5613334740597997\n",
      "-----------------------------------------\n",
      "testing takes 422.158 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 125.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 50, **125**, 200\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.628 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.36706662111878396\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.3664355911269784\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4898026166826487\n",
      "[proc 0][Train](20000/100000) average loss: 0.4284346187740564\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.4901630429238081\n",
      "[proc 0][Train](20000/100000) average regularization: 0.026329864349495618\n",
      "[proc 0][Train] 20000 steps take 680.981 seconds\n",
      "[proc 0]sample: 72.353, forward: 243.351, backward: 82.477, update: 281.609\n",
      "[proc 1][Train](20000/100000) average loss: 0.42829931693971157\n",
      "[proc 1][Train](20000/100000) average regularization: 0.02626844566403888\n",
      "[proc 1][Train] 20000 steps take 680.983 seconds\n",
      "[proc 1]sample: 71.971, forward: 244.638, backward: 76.872, update: 254.805\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.33622898741364476\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.33636710212528703\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.45864450567364695\n",
      "[proc 1][Train](40000/100000) average loss: 0.3974367466107011\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4585302133321762\n",
      "[proc 1][Train](40000/100000) average regularization: 0.03180509959366173\n",
      "[proc 0][Train](40000/100000) average loss: 0.3974486578330398\n",
      "[proc 1][Train] 20000 steps take 684.767 seconds\n",
      "[proc 1]sample: 71.258, forward: 241.216, backward: 76.533, update: 286.954\n",
      "[proc 0][Train](40000/100000) average regularization: 0.03182492801928893\n",
      "[proc 0][Train] 20000 steps take 684.790 seconds\n",
      "[proc 0]sample: 71.653, forward: 241.405, backward: 77.542, update: 291.727\n",
      "[1]Valid average MRR: 0.5580403691133157\n",
      "[1]Valid average MR: 79.46838421049048\n",
      "[1]Valid average HITS@1: 0.46178255037213073\n",
      "[1]Valid average HITS@3: 0.6128376584023915\n",
      "[1]Valid average HITS@10: 0.7364136540988853\n",
      "[proc 1]validation take 353.578 seconds:\n",
      "[0]Valid average MRR: 0.5580008677142926\n",
      "[0]Valid average MR: 80.78246717873292\n",
      "[0]Valid average HITS@1: 0.4619082638775399\n",
      "[0]Valid average HITS@3: 0.6125694557934303\n",
      "[0]Valid average HITS@10: 0.7361973634036063\n",
      "[proc 0]validation take 356.890 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3274699492707849\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3274751476973295\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.45114183987826106\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.45119006725400684\n",
      "[proc 1][Train](60000/100000) average loss: 0.3893058945402503\n",
      "[proc 0][Train](60000/100000) average loss: 0.38933260756284\n",
      "[proc 1][Train](60000/100000) average regularization: 0.033214262520335616\n",
      "[proc 0][Train](60000/100000) average regularization: 0.033231590505875645\n",
      "[proc 1][Train] 20000 steps take 893.631 seconds\n",
      "[proc 1]sample: 69.892, forward: 253.338, backward: 74.231, update: 138.595\n",
      "[proc 0][Train] 20000 steps take 893.626 seconds\n",
      "[proc 0]sample: 68.928, forward: 248.743, backward: 70.467, update: 124.148\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3225444784924388\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4472634916961193\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3226192030027509\n",
      "[proc 0][Train](80000/100000) average loss: 0.3849039852336049\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03409471263941377\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.44709751193970443\n",
      "[proc 0][Train] 20000 steps take 533.894 seconds\n",
      "[proc 0]sample: 70.122, forward: 259.049, backward: 74.777, update: 125.261\n",
      "[proc 1][Train](80000/100000) average loss: 0.3848583575546741\n",
      "[proc 1][Train](80000/100000) average regularization: 0.03407758088745177\n",
      "[proc 1][Train] 20000 steps take 533.919 seconds\n",
      "[proc 1]sample: 69.445, forward: 261.179, backward: 75.698, update: 126.572\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.3192993811950088\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.31921850164830684\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4445017344608903\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4444771364837885\n",
      "[proc 1][Train](100000/100000) average loss: 0.38190055795013905\n",
      "[proc 0][Train](100000/100000) average loss: 0.38184781900644305\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03470921969469637\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03472552032638341\n",
      "[proc 1][Train] 20000 steps take 541.014 seconds\n",
      "[proc 1]sample: 71.695, forward: 264.026, backward: 76.908, update: 128.078\n",
      "[proc 0][Train] 20000 steps take 541.037 seconds\n",
      "[proc 0]sample: 68.425, forward: 253.963, backward: 74.714, update: 128.683\n",
      "[0]Valid average MRR: 0.5897058289966878\n",
      "[0]Valid average MR: 91.26739118592363\n",
      "[0]Valid average HITS@1: 0.5000783080023969\n",
      "[0]Valid average HITS@3: 0.6425988723647655\n",
      "[0]Valid average HITS@10: 0.7535681211526938\n",
      "[proc 0]validation take 341.405 seconds:\n",
      "[1]Valid average MRR: 0.5906692746677442\n",
      "[1]Valid average MR: 89.94371735770171\n",
      "[1]Valid average HITS@1: 0.5007898840368522\n",
      "[1]Valid average HITS@3: 0.6441708600883853\n",
      "[1]Valid average HITS@10: 0.7541451888571876\n",
      "[proc 1]validation take 347.073 seconds:\n",
      "proc 1 takes 3681.387 seconds\n",
      "proc 0 takes 3681.401 seconds\n",
      "training takes 3681.923058986664 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_13\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5885357752708703\n",
      "Test average MR : 91.78944142941774\n",
      "Test average HITS@1 : 0.4982397842799458\n",
      "Test average HITS@3 : 0.6424600121206343\n",
      "Test average HITS@10 : 0.7526982030138162\n",
      "-----------------------------------------\n",
      "testing takes 361.461 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 125.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 15\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 50, **125**, 200\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.589 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.34078783599585294\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.34002403313070534\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.485973194898665\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.48596088933050635\n",
      "[proc 1][Train](20000/100000) average loss: 0.4133805154725909\n",
      "[proc 0][Train](20000/100000) average loss: 0.4129924612134695\n",
      "[proc 1][Train](20000/100000) average regularization: 0.030852843632665463\n",
      "[proc 1][Train] 20000 steps take 528.581 seconds\n",
      "[proc 1]sample: 70.187, forward: 253.892, backward: 76.058, update: 125.943\n",
      "[proc 0][Train](20000/100000) average regularization: 0.030790989415952937\n",
      "[proc 0][Train] 20000 steps take 528.604 seconds\n",
      "[proc 0]sample: 71.126, forward: 253.048, backward: 72.441, update: 124.353\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.31351730078756807\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.44822603252083065\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.31415210523605347\n",
      "[proc 0][Train](40000/100000) average loss: 0.38087166666984557\n",
      "[proc 0][Train](40000/100000) average regularization: 0.034724384064041075\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4477640679612756\n",
      "[proc 0][Train] 20000 steps take 516.816 seconds\n",
      "[proc 0]sample: 71.046, forward: 248.607, backward: 72.689, update: 119.843\n",
      "[proc 1][Train](40000/100000) average loss: 0.38095808654427527\n",
      "[proc 1][Train](40000/100000) average regularization: 0.03474221411887556\n",
      "[proc 1][Train] 20000 steps take 516.832 seconds\n",
      "[proc 1]sample: 69.754, forward: 243.947, backward: 75.700, update: 119.264\n",
      "[1]Valid average MRR: 0.5952203216491456\n",
      "[1]Valid average MR: 99.34394342796054\n",
      "[1]Valid average HITS@1: 0.5047563275839763\n",
      "[1]Valid average HITS@3: 0.6500234922407512\n",
      "[1]Valid average HITS@10: 0.7594258360173503\n",
      "[proc 1]validation take 389.538 seconds:\n",
      "[0]Valid average MRR: 0.5955153132884624\n",
      "[0]Valid average MR: 102.52353325706815\n",
      "[0]Valid average HITS@1: 0.5048891431061721\n",
      "[0]Valid average HITS@3: 0.650851854878248\n",
      "[0]Valid average HITS@10: 0.7594105790706542\n",
      "[proc 0]validation take 391.989 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3083926515325904\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.30801598968356847\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4418521017089486\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4420903257727623\n",
      "[proc 1][Train](60000/100000) average loss: 0.3751223766490817\n",
      "[proc 0][Train](60000/100000) average loss: 0.3750531577423215\n",
      "[proc 1][Train](60000/100000) average regularization: 0.03600549520049244\n",
      "[proc 0][Train](60000/100000) average regularization: 0.03598860322572291\n",
      "[proc 1][Train] 20000 steps take 901.463 seconds\n",
      "[proc 1]sample: 68.883, forward: 246.346, backward: 75.766, update: 117.213\n",
      "[proc 0][Train] 20000 steps take 901.477 seconds\n",
      "[proc 0]sample: 68.257, forward: 248.510, backward: 71.378, update: 114.156\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3056440485984087\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.30534784486442806\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.43893327269256116\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4390124497398734\n",
      "[proc 1][Train](80000/100000) average loss: 0.372288660633564\n",
      "[proc 0][Train](80000/100000) average loss: 0.3721801473647356\n",
      "[proc 1][Train](80000/100000) average regularization: 0.03672833848502487\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03671192858573049\n",
      "[proc 1][Train] 20000 steps take 505.685 seconds\n",
      "[proc 1]sample: 69.092, forward: 247.668, backward: 75.701, update: 112.897\n",
      "[proc 0][Train] 20000 steps take 505.685 seconds\n",
      "[proc 0]sample: 67.904, forward: 246.302, backward: 70.919, update: 108.801\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.3038727481082082\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.30365613290816545\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.436941259932518\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4370043839737773\n",
      "[proc 1][Train](100000/100000) average loss: 0.3704070039793849\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03721313813664019\n",
      "[proc 0][Train](100000/100000) average loss: 0.3703302585035563\n",
      "[proc 1][Train] 20000 steps take 504.437 seconds\n",
      "[proc 1]sample: 68.966, forward: 247.677, backward: 75.532, update: 111.933\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03719779531396925\n",
      "[proc 0][Train] 20000 steps take 504.438 seconds\n",
      "[proc 0]sample: 67.491, forward: 246.163, backward: 70.489, update: 107.563\n",
      "[0]Valid average MRR: 0.6116322570059144\n",
      "[0]Valid average MR: 113.25871261644059\n",
      "[0]Valid average HITS@1: 0.5251232499863813\n",
      "[0]Valid average HITS@3: 0.665675900201558\n",
      "[0]Valid average HITS@10: 0.7664106335457863\n",
      "[proc 0]validation take 371.892 seconds:\n",
      "[1]Valid average MRR: 0.6126477150112932\n",
      "[1]Valid average MR: 110.57705454966396\n",
      "[1]Valid average HITS@1: 0.5267879638015212\n",
      "[1]Valid average HITS@3: 0.6655385851542657\n",
      "[1]Valid average HITS@10: 0.7667833334468224\n",
      "[proc 1]validation take 372.020 seconds:\n",
      "proc 1 takes 3329.018 seconds\n",
      "proc 0 takes 3329.036 seconds\n",
      "training takes 3329.5694546699524 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_14\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.6100632913513323\n",
      "Test average MR : 114.21839612684448\n",
      "Test average HITS@1 : 0.5235892739195271\n",
      "Test average HITS@3 : 0.6633425713449138\n",
      "Test average HITS@10 : 0.7650503551073493\n",
      "-----------------------------------------\n",
      "testing takes 391.615 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 125.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 16\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 50, 125, **200**\n",
    "\n",
    "- lr: **0.01**, 0.05, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.387 seconds\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.42736170771718024\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.42828456127494574\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.6213904811173677\n",
      "[proc 0][Train](20000/100000) average loss: 0.5243760943755508\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.6227110586583614\n",
      "[proc 0][Train](20000/100000) average regularization: 0.02613266706657596\n",
      "[proc 0][Train] 20000 steps take 649.868 seconds\n",
      "[proc 0]sample: 70.740, forward: 234.175, backward: 75.825, update: 268.837\n",
      "[proc 1][Train](20000/100000) average loss: 0.5254978100076317\n",
      "[proc 1][Train](20000/100000) average regularization: 0.026011899674963205\n",
      "[proc 1][Train] 20000 steps take 649.868 seconds\n",
      "[proc 1]sample: 69.429, forward: 245.786, backward: 73.185, update: 120.509\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.421043558588624\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.42061819447577\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.5678397645622492\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.5681663417115808\n",
      "[proc 0][Train](40000/100000) average loss: 0.4944416615664959\n",
      "[proc 1][Train](40000/100000) average loss: 0.4943922682657838\n",
      "[proc 0][Train](40000/100000) average regularization: 0.03128098141262308\n",
      "[proc 1][Train](40000/100000) average regularization: 0.031271253756992516\n",
      "[proc 0][Train] 20000 steps take 629.730 seconds\n",
      "[proc 0]sample: 70.587, forward: 234.868, backward: 74.656, update: 249.315\n",
      "[proc 1][Train] 20000 steps take 629.715 seconds\n",
      "[proc 1]sample: 68.933, forward: 240.682, backward: 74.286, update: 115.995\n",
      "[1]Valid average MRR: 0.3116734664465587\n",
      "[1]Valid average MR: 157.2444657047332\n",
      "[1]Valid average HITS@1: 0.2301728892732386\n",
      "[1]Valid average HITS@3: 0.3398237741476402\n",
      "[1]Valid average HITS@10: 0.46485696970522344\n",
      "[proc 1]validation take 341.139 seconds:\n",
      "[0]Valid average MRR: 0.31014350917850453\n",
      "[0]Valid average MR: 159.70705997712045\n",
      "[0]Valid average HITS@1: 0.2287070327395544\n",
      "[0]Valid average HITS@3: 0.3384846380127472\n",
      "[0]Valid average HITS@10: 0.4626504875524323\n",
      "[proc 0]validation take 343.115 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.4237145447552204\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.42328905340582135\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.5517550538361072\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.5518555950954557\n",
      "[proc 0][Train](60000/100000) average loss: 0.48773479917645457\n",
      "[proc 1][Train](60000/100000) average loss: 0.48757232429236175\n",
      "[proc 0][Train](60000/100000) average regularization: 0.03258838368393481\n",
      "[proc 0][Train] 20000 steps take 856.628 seconds\n",
      "[proc 0]sample: 69.584, forward: 237.555, backward: 72.472, update: 127.999\n",
      "[proc 1][Train](60000/100000) average regularization: 0.0325887043274939\n",
      "[proc 1][Train] 20000 steps take 856.633 seconds\n",
      "[proc 1]sample: 67.496, forward: 247.977, backward: 71.658, update: 125.201\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.42285749653577803\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.5436412912130356\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.4234184665545821\n",
      "[proc 1][Train](80000/100000) average loss: 0.483249393928051\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.543536149699986\n",
      "[proc 1][Train](80000/100000) average regularization: 0.033289132270775734\n",
      "[proc 0][Train](80000/100000) average loss: 0.48347730817347767\n",
      "[proc 1][Train] 20000 steps take 501.990 seconds\n",
      "[proc 1]sample: 66.258, forward: 241.109, backward: 68.322, update: 122.340\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03329074301887303\n",
      "[proc 0][Train] 20000 steps take 502.002 seconds\n",
      "[proc 0]sample: 69.389, forward: 235.793, backward: 73.088, update: 122.261\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.4221345439940691\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.42154118788838385\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.5381833372369409\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.5382209143102169\n",
      "[proc 0][Train](100000/100000) average loss: 0.48015894064605236\n",
      "[proc 1][Train](100000/100000) average loss: 0.4798810510724783\n",
      "[proc 0][Train](100000/100000) average regularization: 0.033791002001799644\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03378840064574033\n",
      "[proc 0][Train] 20000 steps take 508.585 seconds\n",
      "[proc 0]sample: 70.804, forward: 241.834, backward: 73.244, update: 121.587\n",
      "[proc 1][Train] 20000 steps take 508.594 seconds\n",
      "[proc 1]sample: 66.937, forward: 247.005, backward: 68.258, update: 121.994\n",
      "[1]Valid average MRR: 0.332514993750518\n",
      "[1]Valid average MR: 140.62108377537334\n",
      "[1]Valid average HITS@1: 0.2472983923136112\n",
      "[1]Valid average HITS@3: 0.3630981158541983\n",
      "[1]Valid average HITS@10: 0.4941984379362237\n",
      "[proc 1]validation take 332.357 seconds:\n",
      "[0]Valid average MRR: 0.330369024290014\n",
      "[0]Valid average MR: 142.8575679577273\n",
      "[0]Valid average HITS@1: 0.245471754644005\n",
      "[0]Valid average HITS@3: 0.3604891866862777\n",
      "[0]Valid average HITS@10: 0.49164828130958216\n",
      "[proc 0]validation take 339.405 seconds:\n",
      "proc 0 takes 3486.218 seconds\n",
      "proc 1 takes 3486.204 seconds\n",
      "training takes 3486.6982605457306 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_15\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.3302233721851022\n",
      "Test average MR : 141.66526280667588\n",
      "Test average HITS@1 : 0.2450632247696739\n",
      "Test average HITS@3 : 0.36069271468162906\n",
      "Test average HITS@10 : 0.4923258680212724\n",
      "-----------------------------------------\n",
      "testing takes 353.544 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 200.0 --lr 0.01 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 17\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 50, 125, **200**\n",
    "\n",
    "- lr: 0.01, **0.05**, 0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.456 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.384798736384511\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.3842130461901426\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.5178855679437518\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.5184645877033472\n",
      "[proc 1][Train](20000/100000) average loss: 0.4513421521693468\n",
      "[proc 0][Train](20000/100000) average loss: 0.4513388168245554\n",
      "[proc 1][Train](20000/100000) average regularization: 0.0309519872082863\n",
      "[proc 0][Train](20000/100000) average regularization: 0.030957415429875255\n",
      "[proc 1][Train] 20000 steps take 517.452 seconds\n",
      "[proc 1]sample: 70.683, forward: 242.600, backward: 72.741, update: 124.536\n",
      "[proc 0][Train] 20000 steps take 517.453 seconds\n",
      "[proc 0]sample: 70.033, forward: 248.694, backward: 75.085, update: 119.362\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.35433802208304405\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4745584237679839\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.35468510735183956\n",
      "[proc 0][Train](40000/100000) average loss: 0.41444822291731837\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4744030100107193\n",
      "[proc 0][Train](40000/100000) average regularization: 0.03586706815585494\n",
      "[proc 0][Train] 20000 steps take 517.195 seconds\n",
      "[proc 0]sample: 69.800, forward: 248.649, backward: 74.792, update: 115.416\n",
      "[proc 1][Train](40000/100000) average loss: 0.4145440586730838\n",
      "[proc 1][Train](40000/100000) average regularization: 0.03586424759626389\n",
      "[proc 1][Train] 20000 steps take 517.211 seconds\n",
      "[proc 1]sample: 71.188, forward: 246.555, backward: 74.330, update: 122.512\n",
      "[1]Valid average MRR: 0.4980004667325966\n",
      "[1]Valid average MR: 97.42947561233036\n",
      "[1]Valid average HITS@1: 0.39949066098313324\n",
      "[1]Valid average HITS@3: 0.5481250468142479\n",
      "[1]Valid average HITS@10: 0.684363019808385\n",
      "[proc 1]validation take 348.311 seconds:\n",
      "[0]Valid average MRR: 0.4973945569889898\n",
      "[0]Valid average MR: 99.08897491420167\n",
      "[0]Valid average HITS@1: 0.39891798768862013\n",
      "[0]Valid average HITS@3: 0.5480198289480852\n",
      "[0]Valid average HITS@10: 0.6834620308329248\n",
      "[proc 0]validation take 351.749 seconds:\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3415260378494859\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3417117519453168\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.4623824883878231\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.46226982246637344\n",
      "[proc 0][Train](60000/100000) average loss: 0.40195426313728094\n",
      "[proc 1][Train](60000/100000) average loss: 0.40199078717529774\n",
      "[proc 0][Train](60000/100000) average regularization: 0.03727389031983912\n",
      "[proc 1][Train](60000/100000) average regularization: 0.03726776823159307\n",
      "[proc 0][Train] 20000 steps take 870.240 seconds\n",
      "[proc 0]sample: 69.591, forward: 242.666, backward: 74.042, update: 129.081\n",
      "[proc 1][Train] 20000 steps take 870.226 seconds\n",
      "[proc 1]sample: 66.909, forward: 246.243, backward: 72.801, update: 127.620\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.3344841692209244\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.3346021868497133\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.45595925842672586\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.45609044816643\n",
      "[proc 0][Train](80000/100000) average loss: 0.3952217137888074\n",
      "[proc 0][Train](80000/100000) average regularization: 0.03790463745445013\n",
      "[proc 1][Train](80000/100000) average loss: 0.3953463174745441\n",
      "[proc 0][Train] 20000 steps take 517.090 seconds\n",
      "[proc 0]sample: 69.869, forward: 244.010, backward: 74.153, update: 128.739\n",
      "[proc 1][Train](80000/100000) average regularization: 0.037899809801764786\n",
      "[proc 1][Train] 20000 steps take 517.093 seconds\n",
      "[proc 1]sample: 64.895, forward: 248.125, backward: 70.070, update: 127.095\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.3296512333482504\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.32955490113794805\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.45176312274336816\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.4518114855259657\n",
      "[proc 1][Train](100000/100000) average loss: 0.3907071780845523\n",
      "[proc 0][Train](100000/100000) average loss: 0.3906831933364272\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03833301784545183\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03833843472860753\n",
      "[proc 1][Train] 20000 steps take 517.630 seconds\n",
      "[proc 1]sample: 64.466, forward: 250.274, backward: 69.801, update: 128.800\n",
      "[proc 0][Train] 20000 steps take 517.636 seconds\n",
      "[proc 0]sample: 69.466, forward: 245.041, backward: 74.199, update: 127.730\n",
      "[0]Valid average MRR: 0.5469951827858731\n",
      "[0]Valid average MR: 107.08743939641553\n",
      "[0]Valid average HITS@1: 0.4516022498229558\n",
      "[0]Valid average HITS@3: 0.6009151822193168\n",
      "[0]Valid average HITS@10: 0.7234297543171542\n",
      "[proc 0]validation take 341.727 seconds:\n",
      "[1]Valid average MRR: 0.5477246486108881\n",
      "[1]Valid average MR: 105.14996220813444\n",
      "[1]Valid average HITS@1: 0.4522324438058792\n",
      "[1]Valid average HITS@3: 0.6013502931423085\n",
      "[1]Valid average HITS@10: 0.7248207439890506\n",
      "[proc 1]validation take 346.262 seconds:\n",
      "proc 1 takes 3285.874 seconds\n",
      "proc 0 takes 3285.874 seconds\n",
      "training takes 3286.423257827759 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_16\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5458366331622487\n",
      "Test average MR : 106.35935978536943\n",
      "Test average HITS@1 : 0.45067310376761066\n",
      "Test average HITS@3 : 0.5989414872971667\n",
      "Test average HITS@10 : 0.7227285726931642\n",
      "-----------------------------------------\n",
      "testing takes 351.035 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 200.0 --lr 0.05 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 18\n",
    "\n",
    "- batch_size: **4096**\n",
    "\n",
    "- neg_sample_size: **256**\n",
    "\n",
    "- hidden_dim: 200, **400**\n",
    "\n",
    "- gamma: 50, 125, **200**\n",
    "\n",
    "- lr: 0.01, 0.05, **0.1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading train triples....\n",
      "Finished. Read 5286834 train triples.\n",
      "Reading valid triples....\n",
      "Finished. Read 293713 valid triples.\n",
      "Reading test triples....\n",
      "Finished. Read 293714 test triples.\n",
      "|Train|: 5286834\n",
      "random partition 5286834 edges into 2 parts\n",
      "part 0 has 2643417 edges\n",
      "part 1 has 2643417 edges\n",
      "/home/luyanfeng/miniconda3/envs/drkg/lib/python3.8/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.\n",
      "  warnings.warn(msg, warn_type)\n",
      "|valid|: 293713\n",
      "|test|: 293714\n",
      "Total initialize time 16.384 seconds\n",
      "[proc 1][Train](20000/100000) average pos_loss: 0.35543275693506005\n",
      "[proc 1][Train](20000/100000) average neg_loss: 0.492905597782135\n",
      "[proc 0][Train](20000/100000) average pos_loss: 0.35480650689899923\n",
      "[proc 1][Train](20000/100000) average loss: 0.42416917731016873\n",
      "[proc 0][Train](20000/100000) average neg_loss: 0.4928983440414071\n",
      "[proc 1][Train](20000/100000) average regularization: 0.031478300169622525\n",
      "[proc 1][Train] 20000 steps take 526.056 seconds\n",
      "[proc 1]sample: 71.211, forward: 252.619, backward: 72.789, update: 124.364\n",
      "[proc 0][Train](20000/100000) average loss: 0.42385242548286917\n",
      "[proc 0][Train](20000/100000) average regularization: 0.031481645508576184\n",
      "[proc 0][Train] 20000 steps take 526.074 seconds\n",
      "[proc 0]sample: 70.674, forward: 242.521, backward: 75.540, update: 124.988\n",
      "[proc 1][Train](40000/100000) average pos_loss: 0.3223879296496511\n",
      "[proc 1][Train](40000/100000) average neg_loss: 0.4536216435968876\n",
      "[proc 0][Train](40000/100000) average pos_loss: 0.3220559559732676\n",
      "[proc 1][Train](40000/100000) average loss: 0.38800478658229115\n",
      "[proc 1][Train](40000/100000) average regularization: 0.03584863364826888\n",
      "[proc 0][Train](40000/100000) average neg_loss: 0.4538218689277768\n",
      "[proc 1][Train] 20000 steps take 520.880 seconds\n",
      "[proc 1]sample: 70.211, forward: 255.716, backward: 75.401, update: 118.756\n",
      "[proc 0][Train](40000/100000) average loss: 0.3879389123424888\n",
      "[proc 0][Train](40000/100000) average regularization: 0.035851185363531116\n",
      "[proc 0][Train] 20000 steps take 520.888 seconds\n",
      "[proc 0]sample: 72.946, forward: 245.082, backward: 75.777, update: 120.107\n",
      "[0]Valid average MRR: 0.5700344630571179\n",
      "[0]Valid average MR: 106.85564089992918\n",
      "[0]Valid average HITS@1: 0.47623862831617364\n",
      "[0]Valid average HITS@3: 0.624894454431552\n",
      "[0]Valid average HITS@10: 0.7434255324944163\n",
      "[proc 0]validation take 400.014 seconds:\n",
      "[1]Valid average MRR: 0.5707346959127806\n",
      "[1]Valid average MR: 104.86673430616177\n",
      "[1]Valid average HITS@1: 0.4771274096570133\n",
      "[1]Valid average HITS@3: 0.6246382535391571\n",
      "[1]Valid average HITS@10: 0.744087786077613\n",
      "[proc 1]validation take 435.102 seconds:\n",
      "[proc 1][Train](60000/100000) average pos_loss: 0.3144108306735754\n",
      "[proc 1][Train](60000/100000) average neg_loss: 0.4458583673268557\n",
      "[proc 0][Train](60000/100000) average pos_loss: 0.3141877174317837\n",
      "[proc 1][Train](60000/100000) average loss: 0.3801345989882946\n",
      "[proc 0][Train](60000/100000) average neg_loss: 0.44584315807670355\n",
      "[proc 1][Train](60000/100000) average regularization: 0.036913430862128734\n",
      "[proc 0][Train](60000/100000) average loss: 0.380015437681973\n",
      "[proc 1][Train] 20000 steps take 956.287 seconds\n",
      "[proc 1]sample: 69.744, forward: 258.512, backward: 76.312, update: 116.303\n",
      "[proc 0][Train](60000/100000) average regularization: 0.036914078318700194\n",
      "[proc 0][Train] 20000 steps take 956.275 seconds\n",
      "[proc 0]sample: 70.791, forward: 245.679, backward: 73.872, update: 116.643\n",
      "[proc 0][Train](80000/100000) average pos_loss: 0.31015855032801626\n",
      "[proc 1][Train](80000/100000) average pos_loss: 0.31028084089607\n",
      "[proc 0][Train](80000/100000) average neg_loss: 0.4418786360785365\n",
      "[proc 0][Train](80000/100000) average loss: 0.37601859318614006\n",
      "[proc 1][Train](80000/100000) average neg_loss: 0.44207980950325726\n",
      "[proc 0][Train](80000/100000) average regularization: 0.037492545221187175\n",
      "[proc 0][Train] 20000 steps take 515.634 seconds\n",
      "[proc 0]sample: 68.915, forward: 247.171, backward: 71.647, update: 107.538\n",
      "[proc 1][Train](80000/100000) average loss: 0.37618032518178224\n",
      "[proc 1][Train](80000/100000) average regularization: 0.037489320958964525\n",
      "[proc 1][Train] 20000 steps take 515.667 seconds\n",
      "[proc 1]sample: 70.433, forward: 253.925, backward: 77.847, update: 113.103\n",
      "[proc 0][Train](100000/100000) average pos_loss: 0.30723728616833684\n",
      "[proc 1][Train](100000/100000) average pos_loss: 0.30741452529728414\n",
      "[proc 0][Train](100000/100000) average neg_loss: 0.43928523066937925\n",
      "[proc 1][Train](100000/100000) average neg_loss: 0.4391802322432399\n",
      "[proc 0][Train](100000/100000) average loss: 0.3732612584397197\n",
      "[proc 1][Train](100000/100000) average loss: 0.3732973788544536\n",
      "[proc 0][Train](100000/100000) average regularization: 0.03788363973796367\n",
      "[proc 1][Train](100000/100000) average regularization: 0.03788005015198141\n",
      "[proc 0][Train] 20000 steps take 520.958 seconds\n",
      "[proc 0]sample: 69.510, forward: 248.815, backward: 71.951, update: 108.303\n",
      "[proc 1][Train] 20000 steps take 520.932 seconds\n",
      "[proc 1]sample: 71.116, forward: 255.665, backward: 77.922, update: 115.895\n",
      "[0]Valid average MRR: 0.6015926580140895\n",
      "[0]Valid average MR: 115.81775685024786\n",
      "[0]Valid average HITS@1: 0.5147661654954513\n",
      "[0]Valid average HITS@3: 0.6540761017595468\n",
      "[0]Valid average HITS@10: 0.7585423816527755\n",
      "[proc 0]validation take 409.683 seconds:\n",
      "[1]Valid average MRR: 0.6017575744962644\n",
      "[1]Valid average MR: 113.84758983228583\n",
      "[1]Valid average HITS@1: 0.514163437902177\n",
      "[1]Valid average HITS@3: 0.6548955786921972\n",
      "[1]Valid average HITS@10: 0.7597560892568962\n",
      "[proc 1]validation take 417.415 seconds:\n",
      "proc 1 takes 3457.238 seconds\n",
      "proc 0 takes 3457.244 seconds\n",
      "training takes 3457.7754940986633 seconds\n",
      "Save model to ckpts/ComplEx_DRKG_17\n",
      "-------------- Test result --------------\n",
      "Test average MRR : 0.5997430343302921\n",
      "Test average MR : 116.29369556779724\n",
      "Test average HITS@1 : 0.511962317083966\n",
      "Test average HITS@3 : 0.6532664428661895\n",
      "Test average HITS@10 : 0.758000980545701\n",
      "-----------------------------------------\n",
      "testing takes 401.553 seconds\n"
     ]
    }
   ],
   "source": [
    "!DGLBACKEND=pytorch dglke_train --dataset DRKG --data_path ./dataset \\\n",
    "--data_files drkg_train.tsv drkg_valid.tsv drkg_test.tsv --format 'raw_udd_hrt' \\\n",
    "--model_name ComplEx \\\n",
    "--batch_size 4096 --neg_sample_size 256 --hidden_dim 400 \\\n",
    "--gamma 200.0 --lr 0.1 --max_step 100000 -adv --regularization_coef 1.00E-07 \\\n",
    "--gpu 0 1 --num_proc 2 --mix_cpu_gpu --async_update --force_sync_interval 1000 \\\n",
    "--valid --test \\\n",
    "--batch_size_eval 128 --neg_sample_size_eval 10000 \\\n",
    "--log_interval 20000 --eval_interval 50000 --num_thread 32"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.16"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
